System capable of detecting failure of component of system and method thereof

ABSTRACT

This disclosure proposes an inventive system capable of testing a component in the system during runtime. The system may comprise: a substrate; a plurality of functional components, of the plurality of functional components being mounted onto the substrate and including a circuitry; a system bus formed with electrically conductive pattern onto the substrate thereby allowing the plurality of functional components to communicate with each other; one or more wrappers, each of the one or more wrappers connected to one of the plurality of functional components; and an in-system component tester (ICT) configured to: select, as a component under test (CUT), at least one functional component, in an idle state, of the plurality of the functional components; and test, via the one or more test wrappers, the at least one functional component selected as the CUT.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of Korean Patent Applications No.10-2020-0189414 filed on Dec. 31, 2020 and No. 10-2021-0181082 filed onDec. 16, 2021, in the Korean Intellectual Property Office, thedisclosure(s) of which is/are incorporated herein by reference.

BACKGROUND OF THE DISCLOSURE Technical Field

The present disclosure relates to a detection of a failure of a systemin a runtime.

Background Art

Artificial intelligence (AI) can refer to intelligence whichartificially imitates human intelligence, to provide intelligence forrecognition, classification, inference, prediction, control/decisionmaking, etc.

Recently, in order to accelerate an operation speed for artificialintelligence (AI), a neural processing unit (NPU) is being developed.

Such NPUs can cooperate with a system having various semiconductorcomponents implemented into a printed circuit board (PCB)-based boardlevel system.

As high integration can be implemented in accordance with thedevelopment of a semiconductor manufacturing process technology, a“system on chip” (SoC) in which various semiconductor components such asprocessors, memories, and peripheral devices are implemented in one chipor a “system in package” (SiP) in which various semiconductor componentssuch as processors, memories, and peripheral devices are implemented inone package is being proposed.

SoC refers to a semiconductor device (chip) which contains the entiresystem in one chip and refers to a technology in which majorsemiconductor elements such as arithmetic units, memory, and dataconversion elements are implemented in one chip. SiP refers to asemiconductor device which contains the entire system in one package andrefers to a technology in which major semiconductor elements such asarithmetic units, memory, and data conversion elements are implementedin one package. That is, a central processing unit (CPU) of a computer,a digital signal processing (DSP) chip, a microcontroller unit (MCU),and the like are integrated in one semiconductor die or package so thatthe chip or the package itself serves as one system. As described above,when a semiconductor device having various functions is combined intoone chip, circuit board space is significantly reduced so that the sizesof various electronics may also be reduced. Further, as compared with atechnology that separately manufactures a plurality of semiconductordevices, a semiconductor device manufacturing cost is significantlylowered so that the unit sales price of the electronics is also lowered.Accordingly, the SoC or SiP techniques which integrate many componentsand/or functions are emerging as a core component technology in theadvanced digital era, which concentrates on high performance, low cost,and small size.

SUMMARY OF THE DISCLOSURE

The inventor of the present disclosure has recognized that when the NPUis integrated in the SoC or SiP, a size of the board substrate isreduced so that a size of the electronics may be reduced as well.

Further, the inventor of the present disclosure has recognized that whenthe NPU is integrated in the SoC or SiP, the manufacturing cost may bereduced as compared with the semiconductor devices which areindividually manufactured.

However, the inventor of the present disclosure has also recognized thatwhen a plurality of semiconductor components for the NPU is integratedin the SoC or SiP, the complexity is significantly increased, whichincreases a defect rate during the manufacturing process. Some defect(s)during the manufacturing process may be found during the testing phasebefore the shipment from the factory, but certain minute defects forsome components which are integrated in the SoC or SiP may be difficultto be found during the testing phase before the factory shipment andthus unsatisfactory products may be handed over to users. Some minutedefects may be gradually amplified due to a fatigue stress or a physicalstress by repeated usage, which can ultimately cause an erroneousoperation of the SoC or SiP.

When the SoC or SiP is implemented in relatively low-cost consumergoods, some types of erroneous operations may not be so problematic.However, the inventor of the present disclosure has recognized that whenthe SoC or SiP is mounted in high-performance mission-critical products,the situation is quite different.

Specifically, the inventor of the present disclosure has recognized aproblem in that when the NPU in the SoC or SiP erroneously operates dueto the failure, defect, or damage, unpredictable artificial intelligence(AI) operation results may be output.

For example, the inventor of the present disclosure has recognized thatwhen the SoC or SiP including the NPU is used for an electronic devicemounted in an autonomous driving vehicle, a drone, an urban air mobility(UAM) vehicle, an unmanned aerial vehicle (UAV) or used for anelectronic device mounted in an AI robot, the unpredictable AI operationresults may be output due to the failure, defect, or damage of the NPU.

Therefore, the inventor of the present disclosure has recognized that itis necessary to propose a method for performing certain types of testingbefore factory shipment, with respect to the SoC or SiP in the runtime.

In other words, the present inventor recognized that certain types ofdefects, with respect to artificial intelligence applications andtechnologies, should potentially be found through specific testingprocedures or methods.

An aspect of this disclosure provides a system capable of testing acomponent in the system during runtime, the system comprising: asubstrate; a plurality of functional components on the substrate and atleast one of the functional components including circuitry; a system busin a form of electrically conductive patterns on the substrate thatallow the plurality of functional components to communicate with eachother; one or more test wrappers, each of the one or more test wrappersconnected to at least one of the plurality of functional components; andan in-system component tester (ICT) configured to: select, as acomponent under test (CUT), at least one functional component, in anidle state, of the plurality of the functional components; and test, viathe one or more test wrappers, the at least one functional componentselected as the CUT.

The ICT may be further configured to: monitor, via the one or more testwrappers, one or more states of the plurality of the functionalcomponents.

The ICT may be further configured to: stop the test with respect to theat least one functional component, based on a detection of a collisionwith an access from the system bus to the at least one functionalcomponent; and allow a connection of the at least one functionalcomponent to return back to the system bus, based on the stop.

The ICT may be further configured to: return to the select step, if theat least one functional component is in the idle state again, after theconnection of the at least one functional component is returned back tothe system bus.

The returning to the selection step may occur after a back-off timerwith respect to the collision expires.

The plurality of functional components may include: one or moreuniversal processing units (UPUs).

The UPU may include: one or more central processing units (CPUs); one ormore graphic processing units (GPUs); and one or more neural processingunits (NPUs) configured to perform operations for an artificial neuralnetwork (ANN) model.

The plurality of functional components may further include one or moreof: at least one memory; at least one memory controller; and at leastone input and output (I/O) controller.

The ICT may be further configured to: instruct, for the testing step,the one or more test wrappers to isolate from the system bus aconnection of the at least one functional component.

The ICT may include at least one or more of: a detector configured tomonitor one or more states of the plurality of the functionalcomponents; a scheduler configured to manage operations of the ICT; agenerator configured to generate test input data; and a testerconfigured to inject the test input data into the CUT and analyze a testresult acquired from the CUT processing the test input data.

The test input data may be a predefined test data or random bit streamswhich are generated based on a seed.

The ICT may be further configured to: perform an analysis of a testresult acquired from the at least one functional component, after thetesting step is completed; and allow a connection of the at least onefunctional component to the system bus or to another system connection,based on the at least one functional component found to be normal afterthe analysis.

The ICT may be further configured to: deactivate the at least onefunctional component, if the at least one functional component is foundto be defective.

The system may further comprise: a field programmable gate array (FPGA)configured to imitate the at least one functional component that isfound to be defective.

The FPGA's address may be revoked and replaced by an address of the atleast one functional component that is found to be defective.

The deactivating step may include revoking an address of the at leastone functional component that is found to be defective or includespowering off the at least one functional component that is found to bedefective.

The deactivating step may include isolating the at least one functionalcomponent being analyzed as defective from the system bus by cutting offa system bus connection to the at least one functional component that isfound to be defective.

The plurality of functional components may include a spare componentthat can replace the at least one functional component that is found tobe defective; and wherein the ICT is further configured to activate thespare component.

The testing step may include a scan test, which is different from afunction test, and wherein, for the scan test, the ICT is furtherconfigured to: connect a plurality of flip-flops in each CUT to eachother, inject test input into at least one flip-flop, and acquire a testresult from operations of combinational logics of the flip-flops toanalyze whether the CUT is defective or normal during runtime.

The plurality of functional components may include a neural processingunit (NPU), the NPU including a plurality of processing element arraysand being configured to select and test at least one processing elementof the plurality of processing element arrays.

The plurality of the test wrappers may structurally isolate the systembus from signal interference.

According to another aspect, this disclosure provides an apparatuscomprising: a plurality of functional components respectively connectedto a system bus via a respective one of a plurality of test wrappers;and an in-system component tester (ICT), operatively connected to thesystem bus and to the test wrappers, configured to perform operationaltesting of at least one functional component to determine whether thefunction component is normal or defective, wherein the operationaltesting is performed during an operational run-time of the apparatus,wherein the system bus, the test wrappers and the ICT are mountedtogether on a same substrate or a same board.

If the at least one functional component is detected to be in an idlestate during the operational run-time of the apparatus, the functionalcomponent may be isolated from the system bus, and the ICT performs theoperational testing on such isolated functional component.

If the functional component is found to be not defective, the functionalcomponent may be returned to the system bus.

If the functional component is found to be defective, such defectivefunctional component may be kept isolated from the system bus.

The plurality of functional components may include a spare componentthat is capable of being activated when the functional component isfound to be defective as a result of the operational testing.

The apparatus may further comprise: a field programmable gate array(FPGA), operatively connected to the system bus and to the ICT, capableof being adapted to perform at least some functionalities of thefunctional component that is found to be defective as a result of theoperational testing.

The plurality of the test wrappers may have an arrangement that providesisolation for the system bus.

Even though the present disclosure mainly describes the SoC, the presentdisclosure is not limited to the SoC, but may be applied to a system inpackage (SIP) or a printed circuit board (PCB)-based board level system.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and other advantages of thepresent disclosure will be more clearly understood from the followingdetailed description taken in conjunction with the accompanyingdrawings.

FIG. 1 is a schematic conceptual view illustrating a neural processingunit according to the present disclosure.

FIG. 2 is a schematic conceptual view illustrating one processingelement of a processing element array applicable to the presentdisclosure.

FIG. 3 is an exemplary view illustrating a modified example of theneural processing unit 100 of FIG. 1.

FIG. 4 is a schematic conceptual view illustrating an exemplaryartificial neural network model.

FIG. 5A is a view illustrating a basic structure of a convolution neuralnetwork.

FIG. 5B is a view illustrating an overall operation of a convolutionneural network.

FIG. 6A is a view illustrating an exemplary architecture of a system onchip (SoC) including an NPU of FIG. 1 or 3.

FIG. 6B is a view illustrating a first modified example of FIG. 6A.

FIG. 6C is a view illustrating a second modified example of FIG. 6A.

FIG. 6D is a diagram illustrating an exemplary architecture of acomputing system including the NPU shown in FIG. 1 or FIG. 3.

FIG. 6E is a view illustrating a first modified example of FIG. 6D.

FIG. 6F is a view illustrating a second modified example of FIG. 6D.

FIG. 7 is a view illustrating an example of scanning a flip-flop.

FIG. 8 is a view illustrating an example in which an architecture forscan test is added in a hardware design.

FIG. 9A is an exemplary view simply illustrating the SoC of FIGS. 6Athrough 6C or a system of FIGS. 6D through 6F from a viewpoint of anoperation.

FIG. 9B is an exemplary view illustrating a configuration for testingthe NPU.

FIG. 10 is an exemplary view illustrating an operation of a wrapper.

FIG. 11 is an exemplary view illustrating an internal configuration ofan ICT.

FIG. 12 is a block diagram specifically illustrating an operation ofmonitoring whether a functional component is in an idle state, by anICT.

FIG. 13 is an exemplary view illustrating an operation between a master,a slave, and an arbiter which operate on a system bus.

FIG. 14 is a view illustrating an example in which a shift register isadded in an SoC chip.

FIG. 15 is an exemplary view illustrating an operation order of an ICT.

FIG. 16 is a block diagram illustrating for easy understanding of a testprocess of an internal memory.

FIG. 17 is an exemplary view illustrating a process of testing afunction using a random number generator.

FIG. 18A is a view illustrating an example of multiple clocks, FIG. 18Bis an exemplary view illustrating an operation of a tester under themultiple clocks, and FIG. 18C is a view illustrating a path of testinput data.

FIG. 19A is a view illustrating an example of a functional component,and FIG. 19B is a view illustrating an example of test input data (forexample, a test vector) injected into a tester in an ICT.

FIG. 20 is a view illustrating a test process using a DFT.

FIG. 21 is a view illustrating an example of shift data and capture dataduring a test process.

FIG. 22 is a view illustrating an example of switching a test mode to anormal operation mode.

FIG. 23 is a view illustrating an example that flip-flops operate on ascan chain.

FIG. 24 is a view illustrating a part of a CUT which operates in anormal operation mode.

FIG. 25 is an exemplary view illustrating a process for simulation.

FIG. 26 is a view illustrating a test architecture for a JPEG imageencoder.

DETAILED DESCRIPTION OF THE EMBODIMENT

Specific structural or step-by-step descriptions for the embodimentsaccording to the concept of the present disclosure disclosed in thepresent specification or application are merely illustrative for thepurpose of describing the embodiments according to the concept of thepresent disclosure. The examples according to the concept of the presentdisclosure may be carried out in various forms and are not interpretedto be limited to the examples described in the present specification orapplication.

Various modifications and changes may be applied to the examples inaccordance with the concept of the present disclosure and the examplesmay have various forms so that the examples will be described in detailin the specification or the application with reference to the drawings.However, it should be understood that the examples according to theconcept of the present disclosure is not limited to the specificexamples, but includes all changes, equivalents, or alternatives whichare included in the technical scope of the present disclosure.

Terminologies such as first and/or second may be used to describevarious components but the components are not limited by the aboveterminologies. The above terminologies are used to distinguish onecomponent from the other component, for example, a first component maybe referred to as a second component without departing from a scope inaccordance with the concept of the present invention and similarly, asecond component may be referred to as a first component.

It should be understood that, when it is described that an element is“coupled” or “connected” to another element, the element may be directlycoupled or directly connected to the other element or coupled orconnected to the other element through a third element. In contrast,when it is described that an element is “directly coupled” or “directlyconnected” to another element, it should be understood that no elementis present therebetween. Other expressions which describe therelationship between components, for example, “between,” “adjacent to,”and “directly adjacent to” should be interpreted in the same manner.

Terminologies used in the present specification are used only todescribe specific examples, and are not intended to limit the presentdisclosure. A singular form may include a plural form if there is noclearly opposite meaning in the context. In the present specification,it should be understood that terms “include” or “have” indicate that afeature, a number, a step, an operation, a component, a part, or acombination thereof described in the specification is present, but donot exclude a possibility of presence or addition of one or more otherfeatures, numbers, steps, operations, components, parts, or combinationsthereof, in advance.

If it is not contrarily defined, all terms used herein includingtechnological or scientific terms have the same meaning as thosegenerally understood by a person with ordinary skill in the art.Terminologies which are defined in a generally used dictionary should beinterpreted to have the same meaning as the meaning in the context ofthe related art but are not interpreted as an ideally or excessivelyformal meaning if it is not clearly defined in this specification.

When the examples are described, a technology which is well known in thetechnical field of the present disclosure and is not directly related tothe present disclosure will not be described. The reason is thatunnecessary description is omitted to clearly transmit the gist of thepresent disclosure without obscuring the gist.

Definition of Terminologies

Here, in order to help the understanding of the disclosure proposed inthe present specification, terminologies used in the presentspecification will be defined briefly hereafter.

NPU is an abbreviation for a neural processing unit and refers to aprocessor specialized for an operation of an artificial neural networkmodel separately from the central processing unit (CPU).

ANN is an abbreviation for an artificial neural network and refers to anetwork which connects nodes in a layered structure by imitating theconnection of the neurons in the human brain through a synapse toimitate the human intelligence.

Information about a structure of an artificial neural network includesinformation about the number of layers, the number of nodes in a layer,a value of each node, information about an operation processing method,and information about a weight matrix which is applied to each node.

Information about data locality of an artificial neural network isinformation which predicts an operation order of an artificial neuralnetwork model which is processed by the neural processing unit based ona data access request order which is requested to a separate memory bythe neural processing unit.

DNN is an abbreviation for a deep neural network and may mean that thenumber of hidden layers of the artificial neural network is increased toimplement higher artificial intelligence.

CNN is an abbreviation for a convolutional neural network and is aneural network which functions similar to the image processing performedin a visual cortex of the human brain. The convolutional neural networkis known to be appropriate for image processing and is known to be easyto extract features of input data and identify the pattern of thefeatures.

Kernel means a weight matrix which is applied to the CNN. Seed is a termin machine learning (ML) that means an initialization state of a pseudorandom number generator. A seed value is set to ensure reproducibilityof the result that an individual would have got by running a particularML algorithm. Namely, the “seed” is a starting point for the sequenceand the guarantee is that if you start from the same seed, you will getthe same sequence of numbers.

Hereinafter, the present disclosure will be described in detail byexplaining examples of the present disclosure with reference to theaccompanying drawings.

FIG. 1 illustrates a neural processing unit according to the presentdisclosure.

A neural processing unit (NPU) 100 illustrated in FIG. 1 is a processorspecialized to perform an operation for an artificial neural network.

The artificial neural network refers to a network in which are collectedartificial neurons which, when various inputs or entry stimulations,multiply a weight by the inputs or stimulations, add the multipliedvalues, and convert a value obtained by additionally adding a deviationusing an active function to transmit. The artificial neural networktrained as described above may be used to output an inference resultfrom input data.

The neural processing unit 100 may be a semiconductor device implementedby an electric/electronic circuit. The electric/electronic circuit mayrefer to a circuit including a large number of electronic elements(transistors, capacitors, etc.). The neural processing unit 100 includesa processing element (PE) array 110 (shown in FIG. 1 to have PE1 throughPE12, but not limited hereto), an NPU internal memory 120, an NPUscheduler 130, and an NPU interface 140. Each of the processing elementarray 110, the NPU internal memory 120, the NPU scheduler 130, and theNPU interface 140 may be a semiconductor circuit to which a large numberof the electronic elements are connected. Therefore, some of theelectronic elements may be difficult to identify or be distinguishedwith the naked eye, but may be identified only by its operation. Forexample, an arbitrary circuit may operate as the processing elementarray 110 or operate as the NPU scheduler 130.

The neural processing unit 100 may include the processing element array110, the NPU internal memory 120 configured to store an artificialneural network model inferred from the processing element array 110, andthe NPU scheduler 130 configured to control the processing element array110 and the NPU internal memory 120 based on data locality informationor information about a structure of the artificial neural network model.Here, the artificial neural network model may include the data localityinformation or the information about the structure of the artificialneural network. The artificial neural network model may refer to an AIrecognition model trained to perform a specific inference function.

The processing element array 110 may perform an operation for anartificial neural network. For example, when input data is input, theprocessing element array 110 may allow the artificial neural network toperform certain learning operations. When the input data is input aftercompleting the learning procedure(s), the processing element array 110may perform an operation of deducing an inference result through theartificial neural network which completes the learning.

The NPU interface 140 may communicate with various components in an ANNdriving device of FIG. 5A, 6A, or 6B, for example, memories, via asystem bus.

For example, the neural processing unit 100 may call data of theartificial neural network model stored in a memory 200 of FIG. 5A, 6A,or 6B, through the NPU interface 140 into the NPU internal memory 120.

The NPU scheduler 130 is configured to control an operation of theprocessing element array 110 and read/write instructions of the NPUinternal memory 120 for an inference operation of the neural processingunit 100.

The NPU scheduler 130 may be configured to analyze the data localityinformation or the information about the structure of the artificialneural network model to control the processing element array 110 and theNPU internal memory 120.

The NPU scheduler 130 may analyze or receive information related to astructure of an artificial neural network model which may operate in theprocessing element array 110. Data of the artificial neural network,which may be included in the artificial neural network model, may storenode data of each layer, placement data locality information of layersor information about the structure, and weight data of each ofconnection networks connecting the nodes of the layers. The data of theartificial neural network may be stored in a memory provided in the NPUscheduler 130 or the NPU internal memory 120. The NPU scheduler 130 mayaccess the memory 200 of FIG. 5A, 6A, or 6B to utilize necessary data.However, it is not limited thereto, so that the data localityinformation or the information about the structure of the artificialneural network model may be generated based on data such as node dataand weight data of the artificial neural network model. The weight datamay also be referred to as a weight kernel. The node data is alsoreferred to as a feature map. For example, data which defines astructure of the artificial neural network model may be generated whenan artificial neural network model is designed or the learning iscompleted, but the present disclosure is not limited thereto.

The NPU scheduler 130 may schedule an operation order of the artificialneural network model based on the data locality information or theinformation about the structure of the artificial neural network model.

The NPU scheduler 130 may acquire a memory address value in which nodedata of a layer of the artificial neural network model and weight dataof the connection network are stored based on the data localityinformation or the information about the structure of the artificialneural network model. For example, the NPU scheduler 130 may acquire thememory address value in which the node data of the layer of theartificial neural network model and the weight data of the connectionnetwork which are stored in the memory are stored. Accordingly, the NPUscheduler 130 may acquire node data of a layer of an artificial neuralnetwork model to be driven, and weight data of a connection network fromthe memory 200, to store the acquired data in the NPU internal memory120. Node data of each layer may have a corresponding memory addressvalue. The weight data of each connection network may have acorresponding memory address value.

The NPU scheduler 130 may schedule an operation order of the processingelement array 110 based on the data locality information or theinformation about the structure of the artificial neural network model,for example, the placement data locality information of layers of theartificial neural network or the information about the structure of theartificial neural network model.

The NPU scheduler 130 performs scheduling operations based on the datalocality information or the information about the structure of theartificial neural network model so that the NPU scheduler may operate ina different way from a scheduling concept of a normal CPU. Thescheduling of the normal CPU operates to provide the highest efficiencyin consideration of at least one among fairness, efficiency, stability,and reaction time. That is, the normal CPU schedules to perform the mostprocessing during the same time in consideration of a priority and anoperation time.

A known CPU uses an algorithm which schedules a task in consideration ofdata such as a priority or an operation processing time of eachprocessing. In contrast, the NPU scheduler 130 may determine aprocessing order based on the data locality information or theinformation about the structure of the artificial neural network model.

Moreover, the NPU scheduler 130 may determine the processing order basedon the data locality information or the information about the structureof the artificial neural network model and/or data locality informationor information about a structure of a neural processing unit 100 to beused.

However, the present disclosure is not limited to the data localityinformation or the information about the structure of the neuralprocessing unit 100. For example, the data locality information or theinformation about the structure of the neural processing unit 100 maydetermine the processing order by utilizing at least one data of amemory size of the NPU internal memory 120, a hierarchy structure of theNPU internal memory 120, number (size) data of processing elements PE1to PE12, and an operator structure of the processing elements PE1 toPE12. That is, the data locality information or the information aboutthe structure of the neural processing unit 100 may include at least onedata of a memory size of the NPU internal memory 120, a hierarchystructure of the NPU internal memory 120, number data of processingelements PE1 to PE12, and an operator structure of the processingelements PE1 to PE12. However, the present disclosure is not limited tothe data locality information or the information about the structure ofthe neural processing unit 100. The memory size of the NPU internalmemory 120 includes information about a memory capacity. The hierarchystructure of the NPU internal memory 120 includes information about aconnection relationship between specific layers for each hierarchystructure. The operator structure of the processing elements PE1 to PE12includes information about components in the processing elements.

The neural processing unit 100 according to an example of the presentdisclosure may include at least one processing element, the NPU internalmemory 120 which stores an artificial neural network model inferred fromthe at least one processing element, and the NPU scheduler 130configured to control the at least one processing element and the NPUinternal memory 120 based on data locality information or informationabout the structure of the artificial neural network model. The NPUscheduler 130 may be configured to be further supplied with the datalocality information or the information about the structure of theneural processing unit 100. Further, the data locality information orthe information about the structure of the neural processing unit 100may include at least one data of a memory size of the NPU internalmemory 120, a hierarchy structure of the NPU internal memory 120, number(size) data of at least one processing element, and an operatorstructure of the at least one processing element.

According to the structure of the artificial neural network model, theoperation for each layer is sequentially performed. That is, when thestructure of the artificial neural network model is confirmed, theoperating order for every layer may be determined. The order of theoperation or an order of the data flow in accordance with the structureof the artificial neural network model may be defined as a data localityof an artificial neural network model at an algorithm level.

When a compiler compiles the artificial neural network model to beexecuted in the neural processing unit 100, the artificial neuralnetwork data locality of the artificial neural network model at a neuralprocessing unit-memory level may be reconstructed.

That is, the data locality of the artificial neural network model at aneural processing unit-memory level may be constructed in accordancewith the compiler, algorithms applied to the artificial neural networkmodel and an operation characteristic of the neural processing unit 100.

For example, even in the same artificial neural network model, theartificial neural network data locality of the artificial neural networkmodel to be processed may be differently configured in accordance with amethod of operating the artificial neural network model by the neuralprocessing unit 100, such as feature map tiling or stationary techniqueof a processing element, the number of processing elements of the neuralprocessing unit 100, a cache memory capacity such as a feature map or aweight in the neural processing unit 100, a memory hierarchy structurein the neural processing unit 100, and an algorithm characteristic of acompiler which determines an order of an operation of the neuralprocessing unit 100 to operate the artificial neural network model. Thisis because even though the same artificial neural network model iscomputed by the above-mentioned factors, the neural processing unit 100may determine differently the order of data necessary at every moment inthe unit of clock cycles.

The compiler may determine the order of data required for a physicaloperation processing by constructing the artificial neural network datalocality of the artificial neural network model at the neural processingunit-memory level in the unit of words.

In other words, the artificial neural network data locality of theartificial neural network model which is present at the neuralprocessing unit-memory level may be defined as information whichpredicts an operating order of the artificial neural network modelprocessed by the neural processing unit 100 based on a data accessrequest order which is requested to the memory 200 by the neuralprocessing unit 100.

The NPU scheduler 130 may be configured to store the data localityinformation or the information about the structure of the artificialneural network.

That is, even though only the data locality information or theinformation about the structure of the artificial neural network of theartificial neural network model is utilized, the NPU scheduler 130 maydetermine a processing order (sequence). That is, the NPU scheduler 130may determine the operating sequence by utilizing the data localityinformation or the information about the structure from an input layerof the artificial neural network to an output layer. For example, aninput layer operation may be scheduled first, and an output layeroperation may be scheduled last. Accordingly, when the NPU scheduler 130is provided with the data locality information or the information aboutthe structure of the artificial neural network model, it is possible toknow all the operating sequences of the artificial neural network model.As a result, it is possible to determine all scheduling orders.

Moreover, the NPU scheduler 130 may determine the processing order byconsidering the data locality information or the information about thestructure of the artificial neural network model and data localityinformation or information about a structure of the neural processingunit 100 and optimize the processing for every determined order.

Accordingly, when the NPU scheduler 130 is provided with all the datalocality information or the information about the structure of theartificial neural network model and data locality information orinformation about a structure of a neural processing unit 100, it ispossible to further improve the operating efficiency of each of thescheduling order determined by the data locality information or theinformation about the structure of the artificial neural network model.For example, the NPU scheduler 130 may acquire connection network datahaving weight data of four artificial neural network layers and threelayers which connect the layers. In this case, a method of schedulingthe processing order based on the data locality information or theinformation about the structure of the artificial neural network modelby the NPU scheduler 130 will be described below with examples.

For example, the NPU scheduler 130 may set input data for an inferenceoperation as node data of a first layer which is an input layer of theartificial neural network model and schedule to perform a multiply andaccumulate (MAC) operation of node data of the first layer and weightdata of a first connection network corresponding to the first layerfirst. However, the examples of the present disclosure are not limitedto the MAC operation and the artificial neural network operation may beperformed by utilizing a multiplier and an adder which may be modifiedin various forms. Hereinafter, for the convenience of description, thecorresponding operation is referred to as a first operation, a result ofthe first operation is referred to as a first operation value, and thecorresponding scheduling may be referred to as first scheduling.

For example, the NPU scheduler 130 may set the first operation value asnode data of a second layer corresponding to the first connectionnetwork and schedule to perform the MAC operation of the node data ofthe second layer and weight data of a second connection networkcorresponding to the second layer after the first scheduling.Hereinafter, for the convenience of description, the correspondingoperation is referred to as a second operation, a result of the secondoperation is referred to as a second operation value, and thecorresponding scheduling may be referred to as second scheduling.

For example, the NPU scheduler 130 may set the second operation value asnode data of a third layer corresponding to the second connectionnetwork and schedule to perform the MAC operation of the node data ofthe third layer and weight data of a third connection networkcorresponding to the third layer during the second scheduling.Hereinafter, for the convenience of description, the correspondingoperation is referred to as a third operation, a result of the thirdoperation is referred to as a third operation value, and thecorresponding scheduling may be referred to as third scheduling.

For example, the NPU scheduler 130 may set the third operation value asnode data of a fourth layer which is an output layer corresponding to athird connection network and schedule to store an inference result whichis stored in the node data of the fourth layer, in the NPU internalmemory 120. Hereinafter, for the convenience of description, thecorresponding scheduling may be referred to as fourth scheduling.

In summary, the NPU scheduler 130 may control the NPU internal memory120 and the processing element array 110 to perform the operation in theorder of the first scheduling, the second scheduling, the thirdscheduling, and the fourth scheduling. That is, the NPU scheduler 130may be configured to control the NPU internal memory 120 and theprocessing element array 110 to perform the operation in accordance withthe set scheduling order.

In summary, the neural processing unit 100 according to the examples ofthe present disclosure may be configured to schedule the processingorder based on structures of the layers of the artificial neural networkand operating order data corresponding to the structure.

For example, the NPU scheduler 130 may be configured to schedule theprocessing order based on the data locality information or theinformation about the structure from the input layer to the output layerof the artificial neural network of the artificial neural network model.

The NPU scheduler 130 may improve an operation operating rate of theneural processing unit and a memory reusability by controlling the NPUinternal memory 120 by utilizing the scheduling order based on the datalocality information or the information about the structure of theartificial neural network model.

In accordance with the characteristic of the artificial neural networkoperation which is driven in the neural processing unit 100 according toexamples of the present disclosure, an operation value of one layer mayserve as input data of a subsequent layer.

Therefore, the neural processing unit 100 controls the NPU internalmemory 120 in accordance with the scheduling order to improve the memoryreusability of the NPU internal memory 120. The reuse of the memory maybe determined by the number of times that the data stored in the memoryis read. For example, after specific data is stored in the memory, whenthe specific data is read only one time and then deleted or overwritten,the memory reusability may become 100%. For example, after specific datais stored in the memory, when the specific data is read four times andthen deleted or overwritten, the memory reusability may become 400%.That is, the memory reusability may be determined as the number of timesof reusing the data which has been stored once. That is, the memoryreuse may refer to the reuse of data stored in the memory or a specificmemory address in which the specific data is stored.

Specifically, when the NPU scheduler 130 is configured to be providedwith the data locality information or the information about thestructure of the artificial neural network model and figure out orderdata in which the operation of the artificial neural network isperformed by the provided data locality information or information aboutthe structure of the artificial neural network model, the NPU scheduler130 recognizes that an operation result of node data of a specific layerof the artificial neural network model and weight data of a specificconnection network serves as node data of a corresponding subsequentlayer.

Accordingly, the NPU scheduler 130 may reuse a value of a memoryaddress, in which the specific operation result is stored, for asubsequent operation. Therefore, the memory reusability may be improved.

For example, the first operation value of the first scheduling is set asnode data of a second layer of the second scheduling. To be morespecific, the NPU scheduler 130 may reset a memory address valuecorresponding to the first operation value of the first schedulingstored in the NPU internal memory 120 as a memory address valuecorresponding to node data of the second layer of the second scheduling.That is, the memory address value may be reused. Accordingly, the NPUscheduler 130 reuses data of the memory address of the first schedulingso that the NPU internal memory 120 may utilize as the second layer nodedata of the second scheduling without a separate memory writingoperation.

For example, the second operation value of the second scheduling is setas node data of a third layer of the third scheduling. To be morespecific, the NPU scheduler 130 may reset a memory address valuecorresponding to the second operation value of the second schedulingstored in the NPU internal memory 120 as a memory address valuecorresponding to node data of the third layer of the third scheduling.That is, the memory address value may be reused. Accordingly, the NPUscheduler 130 reuses data of the memory address of the second schedulingso that the NPU internal memory 120 may utilize the data as the thirdlayer node data of the third scheduling without a separate memorywriting operation.

For example, the third operation value of the third scheduling is set asnode data of a fourth layer of the fourth scheduling. To be morespecific, the NPU scheduler 130 may reset a memory address valuecorresponding to the third operation value of the third schedulingstored in the NPU internal memory 120 as a memory address valuecorresponding to node data of the fourth layer of the fourth scheduling.That is, the memory address value may be reused. Accordingly, the NPUscheduler 130 reuses data of the memory address of the third schedulingso that the NPU internal memory 120 may utilize the data as the fourthlayer node data of the fourth scheduling without a separate memorywriting operation.

Moreover, the NPU scheduler 130 may be configured to determine whetherto reuse the scheduling order and the memory to control the NPU internalmemory 120. In this case, the NPU scheduler 130 analyzes the datalocality information or the information about the structure of theartificial neural network model to provide an efficient scheduling.Further, data required for an operation which is capable of reusing amemory is not repeatedly stored in the NPU internal memory 120 so thatthe memory usage amount may be reduced. Further, the NPU scheduler 130may increase the efficiency of the NPU internal memory 120 bycalculating the memory usage amount which is reduced as much as thememory is reused.

Moreover, the NPU scheduler 130 may be configured to monitor a resourceusage amount of the NPU internal memory 120 and resource usage amountsof the processing elements PE1 to PE12 based on the data localityinformation or the information about the structure of the neuralprocessing unit 100. Therefore, a hardware resource utilizationefficiency of the neural processing unit 100 may be improved.

The NPU scheduler 130 of the neural processing unit 100 according to theexamples of the present disclosure may have an effect of reusing amemory by utilizing the data locality information or the informationabout the structure of the artificial neural network model.

In other words, when the artificial neural network model is a deepneural network, the number of layers and the number of connectionnetworks may be significantly increased so that, in that case, theeffect of the memory reuse may be maximized to a greater degree.

That is, when the neural processing unit 100 does not figure out thedata locality information or the information about the structure of theartificial neural network model and the operating order, the NPUscheduler 130 may not determine whether to reuse the memory of thevalues stored in the NPU internal memory 120. Therefore, the NPUscheduler 130 unnecessarily generates a memory address required for eachprocessing and copies the substantially same data from one memoryaddress to another memory address. Therefore, unnecessary memory readingand writing tasks are generated and duplicate values are stored in theNPU internal memory 120, which may cause a problem in that the memory isunnecessarily wasted.

The processing element array 110 refers to a configuration in which aplurality of processing elements PE1 to PE12 configured to operate nodedata of the artificial neural network and weight data of a connectionnetwork is disposed. Each processing element may include a multiply andaccumulate (MAC) operator and/or an arithmetic logic unit (ALU)operator, but the examples according to the present disclosure are notlimited thereto.

Even though FIG. 1 illustrates a plurality of processing elements as anexample, operators implemented by a plurality of multiplier and addertrees may also be configured to be disposed in parallel in oneprocessing element, instead of the MAC. In this case, the processingelement array 110 may also be referred to as at least one processingelement including a plurality of operators.

The processing element array 110 is configured to include a plurality ofprocessing elements PE1 to PE12. The plurality of processing elementsPE1 to PE12 of FIG. 1 is just an example for the convenience ofdescription and the number of the plurality of processing elements PE1to PE12 is not limited. A size or the number of processing elementarrays 110 may be determined by the number of the plurality ofprocessing elements PE1 to PE12. The size of the processing elementarray 110 may be implemented by an N×M matrix. Here, N and M areintegers greater than zero. The processing element array 110 may includeN×M processing elements. That is, one or more processing elements may beprovided.

A size of the processing element array 110 may be designed inconsideration of the characteristic of the artificial neural networkmodel in which the neural processing unit 100 operates. For additionalexplanation, the number of processing elements may be determined inconsideration of a data size, a required operating speed, and a requiredpower consumption of the artificial neural network model to operate. Thedata size of the artificial neural network model may be determined so asto correspond to the number of layers of the artificial neural networkmodel and a weight data size of each layer.

Accordingly, the size of the processing element array 110 of the neuralprocessing unit 100 according to the examples of the present disclosureis not limited. As the number of processing elements of the processingelement array 110 is increased, a parallel arithmetic ability of theartificial neural network model which operates is increased, but themanufacturing cost and a physical size of the neural processing unit 100may be increased.

For example, the artificial neural network model which operates in theneural processing unit 100 may be an artificial neural network which istrained to detect thirty specific keywords, that is, an AI keywordrecognition model. In this case, the size of the processing elementarray 110 of the neural processing unit 100 may be designed to be 4×3 inconsideration of the characteristic of the operating amount. In otherwords, the neural processing unit 100 may include twelve processingelements. However, it is not limited thereto and the number of theplurality of processing elements PE1 to PE12 may be selected in therange of 8 to 16,384. That is, the examples of the present disclosureare not limited to the number of processing elements.

The processing element array 110 is configured to perform a functionsuch as addition, multiplication, and accumulation required for theartificial neural network operation. In other words, the processingelement array 110 may be configured to perform a multiplication andaccumulation (MAC) operation.

Hereinafter, a first processing element PE1 among the processing elementarray 110 will be explained with an example.

FIG. 2 illustrates one processing element of a processing element arrayapplicable to the present disclosure.

The neural processing unit 100 according to the examples of the presentdisclosure may include the processing element array 110, the NPUinternal memory 120 configured to store an artificial neural networkmodel inferred from the processing element array 110, and the NPUscheduler 130 configured to control the processing element array 110 andthe NPU internal memory 120 based on data locality information orinformation about a structure of the artificial neural network model.The processing element array 110 is configured to perform the MACoperation and the processing element array 110 is configured to quantizeand output the MAC operation result, but the examples of the presentdisclosure are not limited thereto.

The NPU internal memory 120 may store all or a part of the artificialneural network model in accordance with the memory size and the datasize of the artificial neural network model.

The first processing element PE1 may include a multiplier 111, an adder112, an accumulator 113, and a bit quantization unit 114. However, theexamples according to the present disclosure are not limited thereto andthe processing element array 110 may be modified in consideration of theoperation characteristic of the artificial neural network.

The multiplier 111 multiplies input (N) bit data and (M) bit data. Theoperation value of the multiplier 111 is output as (N+M) bit data. Here,N and M are integers greater than zero. A first input unit whichreceives (N) bit data may be configured to receive a value having acharacteristic of a variable and a second input unit which receives (M)bit data may be configured to receive a value having a characteristic ofa constant. When the NPU scheduler 130 distinguishes the variable valueand constant value characteristics, the NPU scheduler 130 may increase amemory reusability of the NPU internal memory 120. However, the inputdata of the multiplier 111 is not limited to the constant value and thevariable value. That is, according to the examples of the presentdisclosure, the input data of the processing element may operate byunderstanding characteristics of the constant value and the variablevalue so that the operating efficiency of the neural processing unit 100may be improved. However, the neural processing unit 100 is not limitedto the feature of the constant value and the variable value of the inputdata.

Here, the value having a characteristic of a variable or the variablerefers that a value of a memory address in which a corresponding valueis stored is updated whenever the entering input data is updated. Forexample, node data of each layer may be a MAC operation value to whichweight data of the artificial neural network model is reflected. Whenobject recognition of moving image data and the like is inferred by theartificial neural network model, the input image is changed at everyframe so that the node data of each layer is changed.

Here, the value having a characteristic of a constant is storedregardless of the update of the entering input data, where the constantmay refer to a value of a memory address in which a corresponding valueis stored. For example, even though the object recognition of the movingimage data and the like is inferred by the artificial neural networkmodel with unique inference determination criteria of the artificialneural network model, the weight data of the connection network may notbe changed.

That is, the multiplier 111 may be configured to receive one variableand one constant. For additional explanation, a variable value input tothe first input unit may be node data of a layer of the artificialneural network and the node data may be input data of an input layer ofthe artificial neural network, an accumulated value of a hidden layer,and an accumulated value of an output layer. The constant value which isinput to the second input unit may be weight data of the connectionnetwork of the artificial neural network.

The NPU scheduler 130 may be configured to improve the memoryreusability in consideration of the characteristic of the constantvalue.

The variable value is an operation value of each layer and the NPUscheduler 130 may recognize a reusable variable value based on the datalocality information or the information about the structure of theartificial neural network model and control the NPU internal memory 120to reuse the memory.

The constant value is weight data of each connection network and the NPUscheduler 130 may recognize a constant value of a connection networkwhich is repeatedly used based on the data locality information or theinformation about the structure of the artificial neural network modeland control the NPU internal memory 120 to reuse the memory.

That is, the NPU scheduler 130 may be configured to recognize thereusable variable value and the reusable constant value based on thedata locality information or the information about the structure of theartificial neural network model and control the NPU internal memory 120to reuse the memory.

The processing element knows that when 0 is input to one of the firstinput unit and the second input unit of the multiplier 111, theoperation result is 0 even though the operation is not performed.Therefore, the processing element may limit the operation of themultiplier 111 so as not to perform the operation.

For example, when 0 is input to one of the first input unit and thesecond input unit of the multiplier 111, the multiplier 111 may beconfigured to operate in a zero-skipping method.

A bit width of data which is input to the first input unit and thesecond input unit may be determined in accordance with the quantizationof the node data and the weight data of each layer of the artificialneural network model. For example, when the node data of the first layermay be quantized to five bits and the weight data of the first layer maybe quantized to seven bits. In this case, the first input unit may beconfigured to receive five-bit data and the second input unit may beconfigured to receive seven-bit data.

The neural processing unit 100 may control the quantized bit width to beconverted in real time when the quantized data which is stored in theNPU internal memory 120 is input to the input units of the processingelements. That is, each layer may have a different quantized bit widthand when the bit width of the input data is converted, the processingelement may be configured to receive bit width information in real timefrom the neural processing unit 100 and convert the bit width in realtime to generate the input data.

The accumulator 113 accumulates an operation value of the multiplier 111and an operation value of the accumulator 113 using the adder 112 asmany times as the number of (L) loops. Therefore, a bit width of data ofan output unit and an input unit of the accumulator 113 may be output to(N+M+log 2(L)) bits. Here, L is an integer greater than zero.

When the accumulation is completed, the accumulator 113 is applied withan initialization reset to initialize the data stored in the accumulator113 to zero, but the examples according to the present disclosure arenot limited thereto.

The bit quantization unit 114 may reduce the bit width of the dataoutput from the accumulator 113. The bit quantization unit 114 may becontrolled by the NPU scheduler 130. The bit width of the quantized datamay be output to (X) bits. Here, X is an integer greater than zero.According to the above-described configuration, the processing elementarray 110 is configured to perform the MAC operation and the processingelement array 110 may quantize the MAC operation result to output theresult. The quantization may have an effect that the larger the (L)loops, the smaller the power consumption. Further, when the powerconsumption is reduced, the heat generation may also be reduced.Specifically, when the heat generation is reduced, the possibility ofthe erroneous operation of the neural processing unit 100 due to thehigh temperature may be reduced.

Output data (X) bits of the bit quantization unit 114 may serve as nodedata of a subsequent layer or input data of a convolution. When theartificial neural network model is quantized, the bit quantization unit114 may be configured to be supplied with quantized information from theartificial neural network model. However, it is not limited thereto andthe NPU scheduler 130 may also be configured to extract quantizedinformation by analyzing the artificial neural network model.Accordingly, the output data (X) bit is converted to a quantized bitwidth to be output so as to correspond to the quantized data size. Theoutput data (X) bit of the bit quantization unit 114 may be stored inthe NPU internal memory 120 with a quantized bit width.

The processing element array 110 of the neural processing unit 100according to the examples of the present disclosure includes amultiplier 111, an adder 112, an accumulator 113, and a bit quantizationunit 114. The processing element array 110 may reduce data with a bitwidth of (N+M+log 2(L)) bit output from the accumulator 113 to a bitwidth of (X) bits by the bit quantization unit 114. The NPU scheduler130 may control the bit quantization unit 114 to reduce the bit width ofthe output data by a predetermined bit from a least significant bit(LSB) to a most significant bit (MSB). When the bit width of the outputdata is reduced, the power consumption, the operation amount, and thememory usage amount may be reduced. However, when the bit width isreduced below a specific length, there may be a problem in that aninference accuracy of the artificial neural network model sharplydeteriorates. Accordingly, the bit width reduction of the output data,that is, a quantization level may be determined by comparing a reducedamount of the power consumption, the operation amount, and the memoryusage amount with a reduced level of the inference accuracy of theartificial neural network model. The quantization level may bedetermined by determining a target inference accuracy of the artificialneural network model and testing by gradually reducing the bit width.The quantization level may be determined for every operation value ofeach layer.

According to the above-described first processing element PE1, theprocessing element array 110 may reduce the power consumption whileimproving the MAC operation speed by adjusting the bit widths of the (N)bit data and the (M) bit data of the multiplier 111 and reducing the bitwidth of the operation value (X) bits by the bit quantization unit 114.Further, the convolution operation of the artificial neural network maybe more efficiently performed.

The NPU internal memory 120 of the neural processing unit 100 may be amemory system configured in consideration of the MAC operationcharacteristic and the power consumption characteristic of theprocessing element array 110.

For example, the neural processing unit 100 may be configured to reducethe bit width of the operation value of the processing element array 110in consideration of the MAC operation characteristic and the powerconsumption characteristic of the processing element array 110.

The NPU internal memory 120 of the neural processing unit 100 may beconfigured to minimize the power consumption of the neural processingunit 100.

The NPU internal memory 120 of the neural processing unit 100 may be amemory system configured to control the memory at a low power inconsideration of the data size and the operation step of the ongoingartificial neural network model.

The NPU internal memory 120 of the neural processing unit 100 may be alower power memory system configured to reuse a specific memory addressin which weight data is stored in consideration of the data size and theoperation step of the ongoing artificial neural network model.

The neural processing unit 100 may provide various activation functionsto impart non-linearity. For example, the neural processing unit 100 mayprovide a sigmoid function, a hyperbolic tangent function, or an ReLU(Rectified Linear Unit) function. The activation function may beselectively applied after the MAC operation. An operation value to whichthe activation function is applied may be referred to as an activationmap.

FIG. 3 illustrates a modified example of the neural processing unit 100of FIG. 1.

The neural processing unit 100 of FIG. 3 is substantially the same asthe processing unit 100 exemplarily illustrated in FIG. 1, except forthe processing element array 110. Thus, redundant description will beomitted for the convenience of description.

The processing element array 110 exemplarily illustrated in FIG. 3 mayfurther include register files RF1 to RF12 corresponding to processingelements PE1 to PE12 in addition to a plurality of processing elementsPE1 to PE12.

The plurality of processing elements PE1 to PE12 and the plurality ofregister files RF1 to RF12 of FIG. 3 are just an example for theconvenience of description and the number of the plurality of processingelements PE1 to PE12 and the plurality of register files RE1 to RE12 isnot limited.

A size of, or the number of, processing element arrays 110 may bedetermined by the number of the plurality of processing elements PE1 toPE12 and the plurality of register files RF1 to RF12. The size of theprocessing element array 110 and the plurality of register files RF1 toRF12 may be implemented by an N×M matrix. Here, N and M are integersgreater than zero.

An array size of the processing element array 110 may be designed inconsideration of the characteristic of the artificial neural networkmodel in which the neural processing unit 100 operates. For additionalexplanation, the memory size of the register file may be determined inconsideration of a data size, a required operating speed, and a requiredpower consumption of the artificial neural network model to operate.

The register files RF1 to RF12 of the neural processing unit 100 arestatic memory units which are directly connected to the processingelements PE1 to PE12. For example, the register files RF1 to RF12 may beconfigured by flip-flops and/or latches. The register files RF1 to RF12may be configured to store the MAC operation value of the correspondingprocessing elements RF1 to RF12. The register files RF1 to RF12 may beconfigured to provide or be provided with the weight data and/or nodedata to or from the NPU system memory 120.

FIG. 4 illustrates an exemplary artificial neural network (ANN) model.

Hereinafter, an operation of an exemplary artificial neural networkmodel 110 a which may operate in the neural processing unit 100 will beexplained.

The exemplary artificial neural network model 110 a of FIG. 4 may be anartificial neural network which is trained in the neural processing unit100 or trained in a separate machine learning device. The artificialneural network model 110 a may be an artificial neural network which istrained to perform various inference functions such as objectrecognition or voice recognition.

The artificial neural network model 110 a may be a deep neural network(DNN).

However, the artificial neural network model 110 a according to theexamples of the present disclosure is not limited to the deep neuralnetwork.

For example, the artificial neural network model 110 a may beimplemented by a model such as a fully convolutional network (FCN)having VGG, VGG16, DenseNET, and an encoder-decoder structure, a deepneural network (DNN) such as SegNet, DeconvNet, DeepLAB, V3+, or U-net,or SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2,Resnet50, Resnet101, and Inception-v3, but the present disclosure is notlimited thereto. Further, the artificial neural network model 110 a maybe an ensemble model based on at least two different models.

The artificial neural network model 110 a may be stored in the NPUinternal memory 120 of the neural processing unit 100. Alternatively,the artificial neural network model 110 a may be implemented to bestored in a memory 200 and then loaded in the neural processing unit 100during the operation of the artificial neural network model 110 a.

Hereinafter, an inference process by the exemplary artificial neuralnetwork model 110 a, being performed by the neural processing unit 100,will be described with reference to FIG. 4.

The artificial neural network model 110 a may be an exemplary deepneural network model including an input layer 110 a-1, a firstconnection network 110 a-2, a first hidden layer 110 a-3, a secondconnection network 110 a-4, a second hidden layer 110 a-5, a thirdconnection network 110 a-6, and an output layer 110 a-7. However, thepresent disclosure is not limited only to the artificial neural networkmodel illustrated in FIG. 4. The first hidden layer 110 a-3 and thesecond hidden layer 110 a-5 may also be referred to as a plurality ofhidden layers.

The input layer 110 a-1 may exemplarily include input nodes x1 and x2.That is, the input layer 110 a-1 may include information about two inputvalues. The NPU scheduler 130 illustrated in FIG. 1 or FIG. 3 may set amemory address in which information about an input value from the inputlayer 110 a-1 is stored, in the NPU internal memory 120 of FIG. 1 orFIG. 3.

For example, the first connection network 110 a-2 may includeinformation about six weight values for connecting nodes of the inputlayer 110 a-1 to nodes of the first hidden layer 110 a-3, respectively.The NPU scheduler 130 of FIG. 1 or FIG. 3 may set a memory address, inwhich information about a weight value of the first connection network110 a-2 is stored, in the NPU internal memory 120. Each weight value ismultiplied with the input node value, and an accumulated value of themultiplied values is stored in the first hidden layer 110 a-3.

For example, the first hidden layer 110 a-3 may include nodes a1, a2,and a3. That is, the first hidden layer 110 a-3 may include informationabout three node values. The NPU scheduler 130 illustrated in FIG. 1 orFIG. 3 may set a memory address for storing information about a nodevalue of the first hidden layer 110 a-3, in the NPU internal memory 120.

For example, the second connection network 110 a-4 may includeinformation about nine weight values for connecting nodes of the firsthidden layer 110 a-3 to nodes of the second hidden layer 110 a-5,respectively. The NPU scheduler 130 of FIG. 1 or FIG. 3 may set a memoryaddress for storing, in the NPU internal memory 120, information about aweight value of the second connection network 110 a-4. The weight valueof the second connection network 110 a-4 is multiplied with the nodevalue input from the corresponding first hidden layer 110 a-3 and theaccumulated value of the multiplied values is stored in the secondhidden layer 110 a-5.

For example, the second hidden layer 110 a-5 may include nodes b1, b2,and b3. That is, the second hidden layer 110 a-5 may include informationabout three node values. The NPU scheduler 130 may set a memory addressfor storing information about a node value of the second hidden layer110 a-5, in the NPU internal memory 120.

For example, the third connection network 110 a-6 may includeinformation about six weight values which connect nodes of the secondhidden layer 110 a-5 and nodes of the output layer 110 a-7,respectively. The NPU scheduler 130 may set a memory address forstoring, in the NPU internal memory 120, information about a weightvalue of the third connection network 110 a-6. The weight value of thethird connection network 110 a-6 is multiplied with the node value inputfrom the second hidden layer 110 a-5, and the accumulated value of themultiplied values is stored in the output layer 110 a-7.

For example, the output layer 110 a-7 may include nodes y1 and y2. Thatis, the output layer 110 a-7 may include information about two nodevalues. The NPU scheduler 130 may set a memory address for storing, inthe NPU internal memory 120, information about a node value of theoutput layer 110 a-7.

That is, the NPU scheduler 130 may analyze or receive a structure of anartificial neural network model which may operate in the processingelement array 110. Information of the artificial neural network, whichmay be included in the artificial neural network model, may includeinformation about a node value of each layer, placement data localityinformation of layers or information about the structure, andinformation about a weight value of each of connection networksconnecting the nodes of the layers.

The NPU scheduler 130 is provided with data locality information orinformation about a structure of the exemplary artificial neural networkmodel 110 a so that the NPU scheduler 130 may determine an operationorder from input to output of the artificial neural network model 110 a.

Accordingly, the NPU scheduler 130 may set the memory address in whichthe MAC operation values of each layer are stored, in the NPU internalmemory 120, in consideration of the scheduling order. For example, aspecific memory address may be a MAC operation value of an input layer110 a-1 and the first connection network 110 a-2 and also may be inputdata of the first hidden layer 110 a-3. However, the present disclosureis not limited to the MAC operation value, and the MAC operation valuemay also be referred to as an artificial neural network operation value.

At this time, the NPU scheduler 130 knows that the MAC operation resultof the input layer 110 a-1 and the first connection network 110 a-2serves as an input of the first hidden layer 110 a-3 so that it iscontrolled to use the same memory address. That is, the NPU scheduler130 may reuse the MAC operation value based on the data localityinformation or the information about the structure of the artificialneural network model. Therefore, the memory reusable function of the NPUsystem memory 120 may be provided.

That is, the NPU scheduler 130 may store the MAC operation value of theartificial neural network model 110 a in accordance with the schedulingorder in a specific area specified in an arbitrary memory address of theNPU internal memory 120 and use the MAC operation value as input data ofa MAC operation in a subsequent scheduling order in the specific area inwhich the MAC operation value is stored.

MAC Operation from Viewpoint of First Processing Element PE1

The MAC operation will be described in detail from a viewpoint of thefirst processing element PE1. The first processing element PE1 may bedesignated to perform the MAC operation of the node a1 of the firsthidden layer 110 a-3.

First, the first processing element PE1 inputs node x1 data of the inputlayer 110 a-1 to the first input unit of the multiplier 111 and inputsweight data between the node x1 and the node a1 to the second inputunit. The adder 112 adds an operation value of the multiplier 111 and anoperation value of the accumulator 113. At this time, when (L) loops are0, there is no accumulated value so that the accumulated value is 0.Accordingly, the operation value of the adder 112 may be equal to theoperation value of the multiplier 111. At this time, a counter value of(L) loops may be 1.

Second, the first processing element PE1 inputs node x2 data of theinput layer 110 a-1 to the first input unit of the multiplier 111 andinputs weight data between the node x2 and the node a1 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 1, the node x1 data which is calculated in a previous step anda weight multiplied value between the node x1 and the node a1 arestored. Accordingly, the adder 112 generates a MAC operation value ofthe node x1 and the node x2 corresponding to the node a1.

Third, the NPU scheduler 130 may complete the MAC operation of the firstprocessing element PE1 based on the data locality information or theinformation about the structure of the artificial neural network model.At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0.

The bit quantization unit 114 may be appropriately adjusted inaccordance with the accumulated value. In other words, as the (L) loopsare increased, the bit width of the output value may be increased. Atthis time, the NPU scheduler 130 may remove a predetermined lower bit sothat the bit width of the operation value of the first processingelement PE1 is (x) bits.

MAC Operation from Viewpoint of Second Processing Element PE2

The MAC operation will be described in detail from a viewpoint of thesecond processing element PE2. The second processing element PE2 may bedesignated to perform the MAC operation of the node a2 of the firsthidden layer 110 a-3.

First, the second processing element PE2 inputs node x1 data of theinput layer 110 a-1 to the first input unit of the multiplier 111 andinputs weight data between the node x1 and the node a2 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 0, there is no accumulated value so that the accumulated valueis 0. Accordingly, the operation value of the adder 112 may be equal tothe operation value of the multiplier 111. At this time, a counter valueof (L) loops may be 1.

Second, the second processing element PE2 inputs node x2 data of theinput layer 110 a-1 to the first input unit of the multiplier 111 andinputs weight data between the node x2 and the node a2 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 1, the node x1 data which is calculated in a previous step anda weight multiplied value between the node x1 and the node a2 arestored. Accordingly, the adder 112 generates a MAC operation value ofthe node x1 and the node x2 corresponding to the node a2.

Third, the NPU scheduler 130 may complete the MAC operation of the firstprocessing element PE1 based on the data locality information or theinformation about the structure of the artificial neural network model.At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0. The bit quantization unit 114 may be appropriatelyadjusted in accordance with the accumulated value.

MAC Operation from Viewpoint of Third Processing Element PE3

The MAC operation will be described in detail from a viewpoint of thethird processing element PE3. The third processing element PE3 may bedesignated to perform the MAC operation of the node a3 of the firsthidden layer 110 a-3.

First, the third processing element PE3 inputs node x1 data of the inputlayer 110 a-1 to the first input unit of the multiplier 111 and inputsweight data between the node x1 and the node a3 to the second inputunit. The adder 112 adds an operation value of the multiplier 111 and anoperation value of the accumulator 113. At this time, when (L) loops are0, there is no accumulated value so that the accumulated value is 0.Accordingly, the operation value of the adder 112 may be equal to theoperation value of the multiplier 111. At this time, a counter value of(L) loops may be 1.

Second, the third processing element PE3 inputs node x2 data of theinput layer 110 a-1 to the first input unit of the multiplier 111 andinputs weight data between the node x2 and the node a3 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 1, the node x1 data which is calculated in a previous step anda weight multiplied value between the node x1 and the node a3 arestored. Accordingly, the adder 112 generates a MAC operation value ofthe node x1 and the node x2 corresponding to the node a3.

Third, the NPU scheduler 130 may complete the MAC operation of the firstprocessing element PE1 based on the data locality information or theinformation about the structure of the artificial neural network model.At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0. The bit quantization unit 114 may be appropriatelyadjusted in accordance with the accumulated value.

Accordingly, the NPU scheduler 130 of the neural processing unit 100 mayperform the MAC operation of the first hidden layer 110 a-3 bysimultaneously using three processing elements PE1 to PE3.

MAC Operation from Viewpoint of Fourth Processing Element PE4

The MAC operation will be described in detail from a viewpoint of thefourth processing element PE4. The fourth processing element PE4 may bedesignated to perform the MAC operation of the node b1 of the secondhidden layer 110 a-5.

First, the fourth processing element PE4 inputs node a1 data of thefirst hidden layer 110 a-3 to the first input unit of the multiplier 111and inputs weight data between the node a1 and the node b1 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 0, there is no accumulated value so that the accumulated valueis 0. Accordingly, the operation value of the adder 112 may be equal tothe operation value of the multiplier 111. At this time, a counter valueof (L) loops may be 1.

Second, the fourth processing element PE4 inputs node a2 data of thefirst hidden layer 110 a-3 to the first input unit of the multiplier 111and inputs weight data between the node a2 and the node b1 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 1, the node a1 data which is calculated in a previous step anda weight multiplied value between the node a1 and the node b1 arestored. Accordingly, the adder 112 generates a MAC operation value ofthe node a1 and the node a2 corresponding to the node b1. At this time,a counter value of (L) loops may be 2.

Third, the fourth processing element PE4 inputs node a3 data of theinput layer 110 a-1 to the first input unit of the multiplier 111 andinputs weight data between the node a3 and the node b1 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 2, the MAC operation value of the node a1 and the node a2corresponding to the node b1 which is calculated in a previous step isstored. Accordingly, the adder 112 generates a MAC operation value ofthe node a1, the node a2, and the node a3 corresponding to the node b1.

Fourth, the NPU scheduler 130 may complete the MAC operation of thefirst processing element PE1 based on the data locality information orthe information about the structure of the artificial neural networkmodel. At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0. The bit quantization unit 114 may be appropriatelyadjusted in accordance with the accumulated value.

MAC Operation from Viewpoint of Fifth Processing Element PE5

The MAC operation will be described in detail from a viewpoint of thefifth processing element PE5. The fifth processing element PE5 may bedesignated to perform the MAC operation of the node b2 of the secondhidden layer 110 a-5.

First, the fifth processing element PE5 inputs node a1 data of the firsthidden layer 110 a-3 to the first input unit of the multiplier 111 andinputs weight data between the node a1 and the node b2 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 0, there is no accumulated value so that the accumulated valueis 0. Accordingly, the operation value of the adder 112 may be equal tothe operation value of the multiplier 111. At this time, a counter valueof (L) loops may be 1.

Second, the fifth processing element PE5 inputs node a2 data of thefirst hidden layer 110 a-3 to the first input unit of the multiplier 111and inputs weight data between the node a2 and the node b2 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 1, the node a1 data which is calculated in a previous step anda weight multiplied value between the node a1 and the node b2 arestored. Accordingly, the adder 112 generates a MAC operation value ofthe node a1 and the node a2 corresponding to the node b2. At this time,a counter value of (L) loops may be 2.

Third, the fifth processing element PE5 inputs node a3 data of the inputlayer 110 a-1 to the first input unit of the multiplier 111 and inputsweight data between the node a3 and the node b2 to the second inputunit. The adder 112 adds an operation value of the multiplier 111 and anoperation value of the accumulator 113. At this time, when (L) loops are2, the MAC operation value of the node a1 and the node a2 correspondingto the node b2 which is calculated in a previous step is stored.Accordingly, the adder 112 generates a MAC operation value of the nodea1, the node a2, and the node a3 corresponding to the node b2.

Fourth, the NPU scheduler 130 may complete the MAC operation of thefirst processing element PE1 based on the data locality information orthe information about the structure of the artificial neural networkmodel. At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0. The bit quantization unit 114 may be appropriatelyadjusted in accordance with the accumulated value.

MAC Operation from Viewpoint of Sixth Processing Element PE6

The MAC operation will be described in detail from a viewpoint of thesixth processing element PE6. The sixth processing element PE6 may bedesignated to perform the MAC operation of the node b3 of the secondhidden layer 110 a-5.

First, the sixth processing element PE6 inputs node a1 data of the firsthidden layer 110 a-3 to the first input unit of the multiplier 111 andinputs weight data between the node a1 and the node b3 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 0, there is no accumulated value so that the accumulated valueis 0. Accordingly, the operation value of the adder 112 may be equal tothe operation value of the multiplier 111. At this time, a counter valueof (L) loops may be 1.

Second, the sixth processing element PE6 inputs node a2 data of thefirst hidden layer 110 a-3 to the first input unit of the multiplier 111and inputs weight data between the node a2 and the node b3 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 1, the node a1 data which is calculated in a previous step anda weight multiplied value between the node a1 and the node b3 arestored. Accordingly, the adder 112 generates a MAC operation value ofthe node a1 and the node a2 corresponding to the node b3. At this time,a counter value of (L) loops may be 2.

Third, the sixth processing element PE6 inputs node a3 data of the inputlayer 110 a-1 to the first input unit of the multiplier 111 and inputsweight data between the node a3 and the node b3 to the second inputunit. The adder 112 adds an operation value of the multiplier 111 and anoperation value of the accumulator 113. At this time, when (L) loops are2, the MAC operation value of the node a1 and the node a2 correspondingto the node b3 which is calculated in a previous step is stored.Accordingly, the adder 112 generates a MAC operation value of the nodea1, the node a2, and the node a3 corresponding to the node b3.

Fourth, the NPU scheduler 130 may complete the MAC operation of thefirst processing element PE1 based on the data locality information orthe information about the structure of the artificial neural networkmodel. At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0. The bit quantization unit 114 may be appropriatelyadjusted in accordance with the accumulated value.

Accordingly, the NPU scheduler 130 of the neural processing unit 100 mayperform the MAC operation of the second hidden layer 110 a-5 bysimultaneously using three processing elements PE4 to PE6.

MAC Operation from Viewpoint of Seventh Processing Element PE7

The MAC operation will be described in detail from a viewpoint of theseventh processing element PE7. The seventh processing element PE7 maybe designated to perform the MAC operation of the node y1 of the outputlayer 110 a-7.

First, the seventh processing element PE7 inputs node b1 data of thesecond hidden layer 110 a-5 to the first input unit of the multiplier111 and inputs weight data between the node b1 and the node y1 to thesecond input unit. The adder 112 adds an operation value of themultiplier 111 and an operation value of the accumulator 113. At thistime, when (L) loops are 0, there is no accumulated value so that theaccumulated value is 0. Accordingly, the operation value of the adder112 may be equal to the operation value of the multiplier 111. At thistime, a counter value of (L) loops may be 1.

Second, the seventh processing element PE7 inputs node b2 data of thesecond hidden layer 110 a-5 to the first input unit of the multiplier111 and inputs weight data between the node b2 and the node y1 to thesecond input unit. The adder 112 adds an operation value of themultiplier 111 and an operation value of the accumulator 113. At thistime, when (L) loops are 1, the node b1 data which is calculated in aprevious step and a weight multiplied value between the node b1 and thenode y1 are stored. Accordingly, the adder 112 generates a MAC operationvalue of the node b1 and the node b2 corresponding to the node y1. Atthis time, a counter value of (L) loops may be 2.

Third, the seventh processing element PE7 inputs node b3 data of theinput layer 110 a-1 to the first input unit of the multiplier 111 andinputs weight data between the node b3 and the node y1 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 2, the MAC operation value of the node b1 and the node b2corresponding to the node y1 which is calculated in a previous step isstored. Accordingly, the adder 112 generates a MAC operation value ofthe node b1, the node b2, and the node b3 corresponding to the node y1.

Fourth, the NPU scheduler 130 may complete the MAC operation of thefirst processing element PE1 based on the data locality information orthe information about the structure of the artificial neural networkmodel. At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0. The bit quantization unit 114 may be appropriatelyadjusted in accordance with the accumulated value.

MAC Operation from Viewpoint of Eighth Processing Element PE8

The MAC operation will be described in detail from a viewpoint of theeighth processing element PE8. The eighth processing element PE8 may bedesignated to perform the MAC operation of the node y2 of the outputlayer 110 a-7.

First, the eighth processing element PE8 inputs node b1 data of thesecond hidden layer 110 a-5 to the first input unit of the multiplier111 and inputs weight data between the node b1 and the node y2 to thesecond input unit. The adder 112 adds an operation value of themultiplier 111 and an operation value of the accumulator 113. At thistime, when (L) loops are 0, there is no accumulated value so that theaccumulated value is 0. Accordingly, the operation value of the adder112 may be equal to the operation value of the multiplier 111. At thistime, a counter value of (L) loops may be 1.

Second, the eighth processing element PE8 inputs node b2 data of thesecond hidden layer 110 a-5 to the first input unit of the multiplier111 and inputs weight data between the node b2 and the node y2 to thesecond input unit. The adder 112 adds an operation value of themultiplier 111 and an operation value of the accumulator 113. At thistime, when (L) loops are 1, the node b1 data which is calculated in aprevious step and a weight multiplied value between the node b1 and thenode y2 are stored. Accordingly, the adder 112 generates a MAC operationvalue of the node b1 and the node b2 corresponding to the node y2. Atthis time, a counter value of (L) loops may be 2.

Third, the eighth processing element PE8 inputs node b3 data of theinput layer 110 a-1 to the first input unit of the multiplier 111 andinputs weight data between the node b3 and the node y2 to the secondinput unit. The adder 112 adds an operation value of the multiplier 111and an operation value of the accumulator 113. At this time, when (L)loops are 2, the MAC operation value of the node b1 and the node b2corresponding to the node y2 which is calculated in a previous step isstored. Accordingly, the adder 112 generates a MAC operation value ofthe node b1, the node b2, and the node b3 corresponding to the node y2.

Fourth, the NPU scheduler 130 may complete the MAC operation of thefirst processing element PE1 based on the data locality information orthe information about the structure of the artificial neural networkmodel. At this time, the initialization reset is input to initialize theaccumulator 113. That is, the counter value of (L) loops may beinitialized to 0. The bit quantization unit 114 may be appropriatelyadjusted in accordance with the accumulated value.

Accordingly, the NPU scheduler 130 of the neural processing unit 100 mayperform the MAC operation of the output layer 110 a-7 by simultaneouslyusing two processing elements PE7 and PE8.

When the MAC operation of the eighth processing element PE8 iscompleted, the inference operation of the artificial neural networkmodel 110 a may be finished. That is, the artificial neural networkmodel 110 a may determine that the inference operation of one frame iscompleted. If the neural processing unit 100 infers moving image data inreal time, image data of a subsequent frame may be input to the inputnodes x1 and x2 of the input layer 110 a-1. At this time, the NPUscheduler 130 may store image data of a subsequent frame in a memoryaddress in which input data of the input layer 110 a-1 is stored. Whenthis process is repeated at every frame, the neural processing unit 100may process the inference operation in real time. Further, a memoryaddress which has been set may be reused.

According to the summary of the artificial neural network model 110 a ofFIG. 4, the NPU scheduler 130 of the neural processing unit 100 maydetermine an operation scheduling order based on the data localityinformation or the information about the structure of the artificialneural network model 110 a, for the purpose of the inference operationof the artificial neural network model 110 a. The NPU scheduler 130 mayset a memory address required for the NPU internal memory 120 based onthe operation scheduling order. The NPU scheduler 130 may set a memoryaddress that reuses the memory, based on the data locality informationor the information about the structure of the artificial neural networkmodel 110 a. The NPU scheduler 130 designates processing elements PE1 toPE8 required for the inference operation to perform the inferenceoperation.

In other words, when the number of weight data connected to one node isincreased to L, the number of (L) loops of the accumulator of theprocessing element may be set to L−1. That is, even though the weightdata of the artificial neural network is increased, the accumulatorincreases the number of times that the accumulator is accumulated toeasily perform the inference operation.

That is, the NPU scheduler 130 of the neural processing unit 100according to the examples of the present disclosure may control theprocessing element array 110 and the NPU internal memory 120 based onthe data locality information and the information about the structure ofthe artificial neural network model including data locality informationand information about a structure of the input layer 110 a-1, the firstconnection network 110 a-2, the first hidden layer 110 a-3, the secondconnection layer 110 a-4, the second hidden layer 110 a-5, the thirdconnection layer 110 a-6, and the output layer 110 a-7.

That is, the NPU scheduler 130 may set memory address valuescorresponding to node data of the input layer 110 a-1, weight data ofthe first connection network 110 a-2, node data of the first hiddenlayer 110 a-3, weight data of the second connection layer 110 a-4, nodedata of the second hidden layer 110 a-5, weight data of the thirdconnection layer 110 a-6, and node data of the output layer 110 a-7 inthe NPU memory system 110.

Hereinafter, the scheduling of the NPU scheduler 130 will be describedin detail. The NPU scheduler 130 may schedule an operation order of theartificial neural network model based on the data locality informationor the information about the structure of the artificial neural networkmodel.

The NPU scheduler 130 may acquire a memory address value in which nodedata of the layer of the artificial neural network model and weight dataof the connection network are stored based on the data localityinformation or the information about the structure of the artificialneural network model.

For example, the NPU scheduler 130 may acquire the memory address valuein which the node data of the layer of the artificial neural networkmodel and the weight data of the connection network which are stored ina main memory are stored. Accordingly, the NPU scheduler 130 may takenode data of a layer of an artificial neural network model to be drivenand weight data of a connection network from the main memory to storethe data in the NPU internal memory 120. Node data of each layer mayhave a corresponding memory address value. The weight data of eachconnection network may have a corresponding memory address value.

The NPU scheduler 130 may schedule an operation order of the processingelement array 110 based on the data locality information or theinformation about the structure of the artificial neural network model,for example, the placement data locality information or the informationabout the structure of layers of the artificial neural network of theartificial neural network model.

For example, the NPU scheduler 130 may acquire weight data having weightvalues of four artificial neural network layers and three layers whichconnect the layers, that is, connection network data. In this case, amethod of scheduling the processing order based on the data localityinformation or the information about the structure of the artificialneural network model by the NPU scheduler 130 will be described belowwith examples.

For example, the NPU scheduler 130 may set input data for an inferenceoperation as node data of a first layer which is an input layer 110 a-1of the artificial neural network model 110 a and schedule to perform amultiply and accumulate (MAC) operation of node data of the first layerand weight data of a first connection network corresponding to the firstlayer first. Hereinafter, for the convenience of description, thecorresponding operation is referred to as a first operation, a result ofthe first operation is referred to as a first operation value, and thecorresponding scheduling may be referred to as first scheduling.

For example, the NPU scheduler 130 may set the first operation value asnode data of a second layer corresponding to the first connectionnetwork and schedule to perform the MAC operation of the node data ofthe second layer and weight data of a second connection networkcorresponding to the second layer after the first scheduling.Hereinafter, for the convenience of description, the correspondingoperation is referred to as a second operation, a result of the secondoperation is referred to as a second operation value, and thecorresponding scheduling may be referred to as second scheduling.

For example, the NPU scheduler 130 may set the second operation value asnode data of a third layer corresponding to the second connectionnetwork and schedule to perform the MAC operation of the node data ofthe third layer and weight data of a third connection networkcorresponding to the third layer during the second scheduling.Hereinafter, for the convenience of description, the correspondingoperation is referred to as a third operation, a result of the thirdoperation is referred to as a third operation value, and thecorresponding scheduling may be referred to as third scheduling.

For example, the NPU scheduler 130 may set the third operation value asnode data of a fourth layer which is an output layer 110 a-7corresponding to a third connection network and schedule to store aninference result which is stored in the node data of the fourth data, inthe NPU internal memory 120. Hereinafter, for the convenience ofdescription, the corresponding scheduling may be referred to as fourthscheduling. The inference result value is transmitted to variouscomponents to be utilized.

For example, when the inference result value is a result value ofdetecting a specific keyword, the neural processing unit 100 transmitsan inference result to the central processing unit to perform anoperation corresponding to a specific keyword.

For example, the NPU scheduler 130 may drive the first to thirdprocessing elements PE1 to PE3 in the first scheduling.

For example, the NPU scheduler 130 may drive the fourth to sixthprocessing elements PE4 to PE6 in the second scheduling.

For example, the NPU scheduler 130 may drive the seventh and eighthprocessing elements PE7 and PE8 in the third scheduling.

For example, the NPU scheduler 130 may output an inference result in thefourth scheduling.

In summary, the NPU scheduler 130 may control the NPU internal memory120 and the processing element array 110 to perform the operation in theorder of the first scheduling, the second scheduling, the thirdscheduling, and the fourth scheduling. That is, the NPU scheduler 130may be configured to control the NPU internal memory 120 and theprocessing element array 110 to perform the operation in accordance withthe set scheduling order.

In summary, the neural processing unit 100 according to the examples ofthe present disclosure may be configured to schedule the processingorder based on structures of the layers of the artificial neural networkand operating order data corresponding to the structure. The order ofprocessing to be scheduled may be at least one. For example, the neuralprocessing unit 100 may predict all operation orders, so that it ispossible to schedule a subsequent operation and it is also possible toschedule an operation in a specific order.

The NPU scheduler 130 may improve a memory reusability by controllingthe NPU internal memory 120 by utilizing the scheduling order based onthe data locality information or the information about the structure ofthe artificial neural network model.

In accordance with the characteristic of the artificial neural networkoperation, which is driven in the neural processing unit 100 accordingto examples of the present disclosure, an operation value of one layermay serve as input data of a subsequent layer.

Therefore, when the neural processing unit 100 controls the NPU internalmemory 120 in accordance with the scheduling order, it is possible toimprove the memory reusability of the NPU internal memory 120.

Specifically, when the NPU scheduler 130 is configured to be providedwith the data locality information or the information about thestructure of the artificial neural network model and figure out an orderin which the operation of the artificial neural network is performed bythe provided data locality information or information about thestructure of the artificial neural network model, the NPU scheduler 130may recognize that an operation result of node data of a specific layerof the artificial neural network model and weight data of a specificconnection network serves as node data of a corresponding subsequentlayer. Accordingly, the NPU scheduler 130 may reuse, for a subsequentoperation, a value of a memory address in which the correspondingoperation result is stored.

For example, the first operation value of the first scheduling is set asnode data of a second layer of the second scheduling. To be morespecific, the NPU scheduler 130 may reset a memory address valuecorresponding to the first operation value of the first schedulingstored in the NPU internal memory 120 as a memory address valuecorresponding to node data of the second layer of the second scheduling.That is, the memory address value may be reused. Accordingly, the NPUscheduler 130 reuses the memory address value of the first scheduling sothat the NPU internal memory 120 may utilize the data as the secondlayer node data of the second scheduling without a separate memorywriting operation.

For example, the second operation value of the second scheduling is setas node data of a third layer of the third scheduling. To be morespecific, the NPU scheduler 130 may reset a memory address valuecorresponding to the second operation value of the second schedulingstored in the NPU internal memory 120 as a memory address valuecorresponding to node data of the third layer of the third scheduling.That is, the memory address value may be reused. Accordingly, the NPUscheduler 130 reuses the memory address value of the second schedulingso that the NPU internal memory 120 may utilize the data as the thirdlayer node data of the third scheduling without a separate memorywriting operation.

For example, the third operation value of the third scheduling is set asnode data of a fourth layer of the fourth scheduling. To be morespecific, the NPU scheduler 130 may reset a memory address valuecorresponding to the third operation value of the third schedulingstored in the NPU internal memory 120 as a memory address valuecorresponding to node data of the fourth layer of the fourth scheduling.That is, the memory address value may be reused. Accordingly, the NPUscheduler 130 reuses the memory address value of the third scheduling sothat the NPU internal memory 120 may utilize the data as the fourthlayer node data of the fourth scheduling without a separate memorywriting operation.

Moreover, the NPU scheduler 130 may be configured to determine whetherto reuse the scheduling order and the memory to control the NPU internalmemory 120. In this case, the NPU scheduler 130 analyzes the datalocality information or the information about the structure of theartificial neural network model to provide an optimized scheduling.Further, data required for an operation which is capable of reusing amemory is not repeatedly stored in the NPU internal memory 120 so thatthe memory usage amount may be reduced. Further, the NPU scheduler 130may optimize the NPU internal memory 120 by calculating the memory usageamount which is reduced as much as the memory is reused.

The neural processing unit 100 according to examples of the presentdisclosure may be configured such that a variable value is input to (N)bit input which is a first input of the first processing element PE1 anda constant value is input to (M) bit input which is a second input.Further, this configuration may be set in the other processing elementsof the processing element array 110 in the same way. That is, one inputof the processing element may be configured to receive a variable valueand the other input may be configured to receive a constant value.Therefore, the number of times of updating data of the constant valuemay be reduced.

At this time, the NPU scheduler 130 utilizes the data localityinformation or the information about the structure of the artificialneural network model 110 a to set node data of the input layer 110 a-1,the first hidden layer 110 a-3, the second hidden layer 110 a-5, and theoutput layer 110 a-7 as variables and set weight data of the firstconnection network 110 a-2, weight data of the second connection network110 a-4, and weight data of the third connection network 110 a-6 asconstants. That is, the NPU scheduler 130 may distinguish the constantvalue and the variable value. However, the present disclosure is notlimited to constant and variable data types, but a value whichfrequently varies is distinguished from a value which does not vary toimprove the reusability of the NPU internal memory 120.

That is, the NPU system memory 120 may be configured to preserve weightdata of connection networks stored in the NPU system memory 120 whilethe inference operation of the neural processing unit 100 is maintained.Therefore, the memory reading and writing operations may be reduced.

That is, the NPU system memory 120 may be configured to reuse the MACoperation value stored in the NPU system memory 120 while the inferenceoperation is maintained.

That is, the number of times of updating data of the memory address inwhich input data (N) bit of the first input unit of the processingelement of the processing element array 110 is stored may be larger thanthe number of times of updating data of the memory address in whichinput data (M) bit of the second input unit is stored. That is, thenumber of times of updating data of the second input unit may be smallerthan the number of times of updating data of the first input unit.

Hereinafter, a convolutional neural network (CNN) which is one type of adeep neural network (DNN) among the artificial neural networks will bemainly described.

The convolutional neural network may be a combination of one or aplurality of convolutional layers, pooling layers, and fully connectedlayers. The CNN has a structure appropriate for the learning oftwo-dimensional data and inference and may be trained by abackpropagation algorithm.

FIG. 5A illustrates a basic structure of a convolution neural network.

Referring to FIG. 5A, the input image may be represented by atwo-dimensional matrix configured by a row with a specific size and acolumn with a specific size. The input image may have a plurality ofchannels and the channel may indicate the number of color components ofan input data image.

The convolution process refers that a convolutional operation with akernel is performed while visiting the input image at a specifiedinterval.

When the convolutional neural network moves from a current layer to anext layer, a weight value between the layers is reflected by theconvolution to transmit the weight value to the next layer.

For example, the convolution is defined by two main parameters and asize of an input image (generally, 1×1, 3×3, and 5×5 matrixes) and adepth (the number of kernels) of an output feature map may be computedby the convolution. The convolution may start from a depth 32, continueto a depth 64, and end at a depth 128 or 256.

The convolution may operate by sliding a window having a 3×3 or 5×5 sizeon a 3D input feature map, stopping in all positions, and extracting a3D patch of a neighboring feature.

The 3D patch may be converted by a 1D vector by a tensor product havingthe same learning weight matrix called a weight. The vector may bespatially reassembled to a 3D output map. All space positions of theoutput feature map may correspond to the same position of the inputfeature map.

The convolutional neural network may include a convolutional layer whichperforms a convolution operation between a kernel (that is, a weightmatrix) which is trained over many iterations of gradient update duringthe learning process and input data. If (m, n) is set as a kernel sizeand W is set as a weight value, the convolutional layer calculates aninner product to perform the convolution of the input data and theweight matrix.

The step size that the kernel slides across the input data is called astep, and the kernel area (m×n) may be called the receptive field. Thesame convolution kernel is applied over different positions of theinput, which may reduce the number of kernels to be learned. This alsoenables position invariant learning and if a significant pattern ispresent in the input, a convolution filter may learn that patternregardless of the position of the sequence.

The convolutional neural network may be adjusted or trained such thatthe input data is connected to a specific output estimate. Theconvolutional neural network may be adjusted using a backpropagationbased on the comparison between the output estimate and the ground truthuntil the output estimate gradually matches or approximates the groundtruth.

The convolutional neural network may be trained by adjusting a weightbetween neurons based on the difference between the ground through andan actual output.

FIG. 5B illustrates an overall operation of a convolution neuralnetwork.

Referring to FIG. 5B, the input image is a two-dimensional matrix thatis 5×5 in size. Further, in FIG. 5B, three nodes, that is, a channel 1,a channel 2, and a channel 3, are used.

First, a convolution operation of the layer 1 will be described.

The input image is convoluted with a kernel 1 for a channel 1 at a firstnode of the layer 1, and a feature map 1 is output as a result. Further,the input image is convoluted with a kernel 2 for a channel 2 at asecond node of the layer 1, and a feature map 2 is output as a result.The input image is convoluted with a kernel 3 for a channel 3 at a thirdnode, and a feature map 3 is output as a result.

Next, a pooling operation of the layer 2 will be described.

The feature map 1, the feature map 2, and the feature map 3 output fromthe layer 1 are input to three nodes of the layer 2. Layer 2 receivesfeature maps output from the layer 1 as inputs to perform the pooling.The pooling may reduce a size or emphasize a specific value in thematrix. The pooling method may include max pooling, average pooling, andminimum pooling. The max pooling is used to collect maximum values in aspecific area of the matrix, and the average pooling is used tocalculate an average in a specific area.

In the example of FIG. 5B, a feature map of a 5×5 matrix is reduced to a4×4 matrix by the pooling.

Specifically, the first node of the layer 2 performs the pooling withthe feature map 1 for the channel 1 as an input, and then outputs a 4×4matrix. The second node of the layer 2 performs the pooling with thefeature map 2 for the channel 2 as an input, and then outputs a 4×4matrix. The third node of the layer 2 performs the pooling with thefeature map 3 for the channel 3 as an input, and then outputs a 4×4matrix.

Next, a convolution operation of the layer 3 will be described.

A first node of the layer 3 receives the output from the first node ofthe layer 2 as an input to perform the convolution with a kernel 4 andoutput a result thereof. A second node of the layer 3 receives theoutput from the second node of the layer 2 as an input to perform theconvolution with a kernel 5 for the channel 2 and outputs a resultthereof. Similarly, a third node of the layer 3 receives the output fromthe third node of the layer 2 as an input to perform the convolutionwith a kernel 6 for the channel 3 and outputs a result thereof.

As described above, the convolution and the pooling are repeated andfinally, as illustrated in FIG. 5A, a fully connected layer may beoutput. The output may be input to the artificial neural network forimage recognition again.

Hereinafter, SoC will be mainly explained, but the disclosure of thepresent specification is not limited to the SoC and the contents of thepresent disclosure is also applicable to a system in package (SIP) or aprinted circuit board (PCB)-based board level system. For example, eachfunctional component is implemented by an independent semiconductor chipand is connected by a system bus which is implemented by an electricallyconductive pattern formed on the PCB.

FIG. 6A illustrates an exemplary architecture of a system on chip (SoC)including an NPU of FIG. 1 or FIG. 3.

Referring to FIG. 6A, an exemplary SoC 1000 includes a plurality offunctional components, a system bus 500, an in-system component tester(ICT) 600, and a plurality of test wrappers 700 a, 700 b, . . . , and700 g, collectively denoted as test wrappers 700.

A (test) wrapper can adapt the interfaces of the components thatcommunicate with each other. Accordingly, the (test) wrapper may also bereferred to as an interface circuit. A so-called NPU test wrapper mayinclude various components, such as interfaces for inputs and outputs,an interface controller, a test wrapper controller, a test accessmechanism, and the like.

The plurality of functional components may include an array 100-1 of NPUcores, an array 200 of central processing unit (CPU) cores, an array 300of graphics processing unit (GPU) cores, an internal memory 400, amemory controller 450, an input/output (I/O) interface 800, and a fieldprogrammable gate array (FPGA) 900.

The examples of the present disclosure are not limited thereto, and atleast some of the plurality of functional components may be removed. Theexamples of the present disclosure are not limited thereto and mayfurther include other functional components other than the plurality offunctional components described above.

The NPU, CPU, and GPU are collectively called universal processing units(UPI), application processing units (APU), or application-dedicatedprocessing units (ADPU).

The individual NPU core in the array 110-1 may refer to the NPU 100 ofFIG. 1 or FIG. 3. In other words, an array 110-a in which the pluralityof NPUs 100 of FIG. 1 or 3 is included is illustrated in FIG. 6.

Similarly, a plurality of CPU cores may be included in the array 200. Aplurality of GPU cores may be included in the array 300.

The array 100-1 of the NPU cores may be connected to the system bus 500via the wrapper 700 a. Similarly, the array 200 of the CPU cores may beconnected to the system bus 500 via the wrapper 700 b. Similarly, thearray 300 of the GPU cores may be connected to the system bus 500 viathe wrapper 700 c.

The internal memory 400 may be connected to the system bus 500 via thewrapper 700 d. The internal memory 400 may be shared by the CPU core,the GPU core, and the NPU core.

The memory controller 450 which is connected to the external memory maybe connected to the system bus 500 via the wrapper 700 e.

The system bus 500 may be implemented by an electrically conductivepattern formed on a semiconductor die. The system bus enables high speedcommunication. For example, the CPU core, the GPU core, and the NPU coremay read out data from the internal memory 400 or write data in theinternal memory 400, by means of the system bus 500. Further, the CPUcore, the GPU core, and the NPU core may read out data from the externalmemory or write data in the external memory by means of the memorycontroller 450.

The ICT 600 may be connected to the system bus 500 by means of adedicated signaling channel. Further, the ICT 600 may be connected tothe plurality of wrappers 700 (700 a-700 g) by means of a dedicatedsignaling channel.

Each wrapper 700 (700 a-700 g) may be connected to the ICT 600 by meansof a dedicated signaling channel. Further, each wrapper 700 may beconnected to the system bus 500 by means of a dedicated signalingchannel. Further, each wrapper 700 may be connected to the respectivefunctional components in the SoC by means of a dedicated signalingchannel.

To this end, each wrapper 700 may be designed to be located between therespective functional components in the SoC and the system bus 500.

For example, a first wrapper 700 a may be connected to the array 100-1of the NPU cores, the system bus 500, and the ICT 600 by means ofdedicated signaling channels. A second wrapper 700 b may be connected tothe array 200 of the CPU cores, the system bus 500, and the ICT 600 bymeans of dedicated signaling channels. A third wrapper 700 c may beconnected to the array 300 of the GPU cores, the system bus 500, and theICT 600 by means of dedicated signaling channels. A fourth wrapper 700 dmay be connected to the internal memory 200, the system bus 500, and theICT 600 by means of dedicated signaling channels. A fifth wrapper 700 emay be connected to the memory controller 450, the system bus 500, andthe ICT 600 by means of dedicated signaling channels. A sixth wrapper700 f may be connected to the I/O interface 800, the system bus 500, andthe ICT 600 by means of dedicated signaling channels. A seventh wrapper700 g may be further connected to the I/O interface 800. It may beconfigured to communicate with the outside through the seventh wrapper700 g.

The ICT 600 may directly monitor the system bus 500 or monitor states ofthe plurality of functional components by means of each wrapper 700.Each functional component may be in an idle state or a busy state.

When a functional component in an idle state is found, the ICT 600 mayselect the corresponding functional component as a component under test(CUT).

If a plurality of functional components are in an idle state, the ICT600 may select any one functional component as a CUT in accordance witha predetermined rule.

If a plurality of functional components are in an idle state, the ICT600 may randomly select any one functional component as a CUT. By doingthis, the ICT 600 may cut off the connection between the functionalcomponent selected as the CUT and the system bus 500 or isolate suchfrom the system bus 500. To this end, the ICT 600 may instruct theparticular wrapper 700 (among 700 a-700 g) connected to the functionalcomponent selected as the CUT to cut off or isolate. To be morespecific, the ICT 600 cuts off the connection between the functionalcomponent selected as the CUT and the system bus 500 by means of thewrapper 700 and then may instruct the wrapper 700 to transmit a signalto the system bus 500, instead of the functional component selected asthe CUT. At this time, the signal which is transmitted to the system bus500 may be a signal which is transmitted to the system bus 500 when thefunctional component selected as the CUT is in an idle state. To thisend, when the functional component selected as the CUT is in an idlestate, the wrapper 700 may monitor (or overhear) and store the signalwhich is transmitted to the system bus 500. The corresponding wrapper700 regenerates the stored signal to transmit the regenerated signal tothe system bus 500. In the meantime, the corresponding wrapper 700 maydetect a signal from the system bus 500.

Thereafter, the ICT 600 may test the functional component selected asthe CUT in order to determine whether any defects are present.

Specifically, the rule may include one or more of a priority ruleaccording to the mission to be performed, a rule for priority betweenfunctional components, a rule according to the presence or absence of aspare for the corresponding functional component, a rule defined by thenumber of tests, and a rule defined by a previous test result.

For example, when the priority rule according to the mission instructsthat an operation by the GPU has a higher priority than the operation bythe CPU, between the CPU and the GPU in the idle state, the GPU may bepreferentially tested. When the rule for priority between functionalcomponents instructs that the priority of the CPU is higher than thepriority of the GPU, between the CPU and the GPU in the idle state, theCPU may be preferentially tested. According to the rule according to thepresence or absence of a spare, when the GPU has three cores and the CPUhas six cores, the GPU having less cores may be preferentially tested.According to the rule defined by the number of tests, when the CPU istested three times and the GPU is tested five times, the CPU which wasis tested may be preferentially tested. According to the rule defined bya previous test result, when as the previous test result for the CPU, anabnormality is discovered and the previous test result for the GPU isnormal, the CPU may be preferentially tested.

When a (signal) collision occurs due to access from the system bus 500to a functional component selected as the CUT at the time of startingthe test or during the test, the ICT 600 may detect such collision.

If so, the ICT 600 may stop (interrupt) the test and drive a back-offtimer with respect to the collision. Here, it can be understood thatother types of test pausing techniques may be used (instead of stoppingor interrupting), and that other types of timers or mechanisms can beemployed (instead of the back-off timer).

The ICT 600, depending upon certain conditions, may reinstate theconnection of the functional component selected as the CUT to the systembus 500.

When the back-off time of the back-off timer for the conflict expires,the ICT 600 may monitor whether the functional components enter the idlestate again. If the functional component enters the idle state again,the ICT 600 may select the functional component as a CUT again, ifnecessary.

If the conflict (or collision) is not detected, the ICT 600 may continuethe test and when the test is completed, analyze the test result.

The test may be for verifying whether a component of the system isdefective in its manufacture, has been compromised, or has broken down.The compromising or the breakdown may be caused by a fatigue stress dueto repeated usage or a physical stress such as heat or electromagneticpulse (EMP). That is, it may be configured to detect a defect based onthe test result.

Hereafter, the test being performed on the NPU will be described in anexemplary manner. As it will be described below, there are two types oftests including a function test and a scan test.

First, when the function test is performed on the NPU, the ICT 600 mayinput a predetermined or particular ANN test model and a test input tothe NPU. When the NPU outputs an inference result for the test inputusing the input ANN test model, the ICT 600 compares an intendedinference result and the inference result from the NPU to analyzewhether the NPU is normal or defective. For example, when the ANN testmodel is a predetermined or particular CNN and the test input is asimple test image, the NPU performs the convolution and the pooling onthe test image using the ANN test model to output a fully connectedlayer.

Next, when the scan test is performed on the NPU, as it will bedescribed below, the ICT 600 may thread the flip-flops in the NPU with ascan chain. The ICT 600 may inject the test input to at least oneflip-flop and acquire a test result from an operation of a combinationallogic of the flip-flop to analyze whether the NPU is defective or normalduring the runtime.

The test performed by the ICT 600 may be used to determine a fairquality before the SoC which is mass-produced in the factory comes out.According to the present disclosure, it is noted that the test fordetermining a fair quality may also be performed during the runtime ofthe SoC. That is, a test for determining a fair quality may be possibleonly before the SoC comes out from the factory. However, according tothe present disclosure, functional components in the idle state arefound from a plurality of functional components in the SoC to besequentially tested so that the fair quality test may be performed onthe SoC in the runtime (i.e., during actual use).

As a test analysis result, when the corresponding functional componentis determined as normal, the ICT 600 returns back the connection withthe functional component to the system bus 500. Specifically, the ICT600 may disconnect the connection between the wrapper 700 and the systembus 500 and resume the connection between the functional component andthe system bus 500. To be more specific, the ICT 600 may initialize thefunctional component to be connected to the system bus 500 and theninstruct the wrapper 700 to stop a signal which is transmitted to thesystem bus 500.

However, if the test analysis result indicates that the component isdefective, the ICT 600 may repeat the test several times in order tofurther check the result.

When as a result of several repeated tests, the functional componentcould be finally determined as being defective, that is, when it isdetermined that the functional component in the SoC is defective in itsmanufacture, has been compromised, or has broken down, the ICT 600 maydeactivate the functional component.

As an alternative, when an error code included in a one-time testanalysis result indicates that the functional component in the SoC isdefective in its manufacture, has been compromised, or has broken down,the ICT 600 may deactivate the functional component.

In order to deactivate the functional component, the ICT 600 may cut-offor disconnect the connection of the functional component determined asdefective to isolate the functional component determined as defectivefrom the system bus 500. Alternatively, in order to deactivate thedefective functional component, the ICT 600 may power off (i.e., turnoff) the functional component. When the functional component is poweredoff, the erroneous operation of the defective functional component isprevented (or minimized) and the power consumption of the SoC may bereduced accordingly.

Further, in order to deactivate the defective functional component, theICT 600 may revoke the address of the functional component on the systembus 500 or transmit a signal for deleting to the system bus 500. Thatis, the ICT 600 may transmit a signal for deleting an address of thedefective functional component to a component having addresses used onthe system bus 500.

When the deactivation is completed, the ICT 600 may determine whetherthere is a spare component that can be used to replace or in lieu of thefunctional component that was found to be defective. Even though a sparemay exist, when the spare is not in an active state, the ICT 600 mayactivate the spare. That is, the ICT 600 may transmit a signal includinga request for updating an address of the activated spare in an addresstable to a component having the table of addresses used on the systembus 500.

When an address on the system bus 500 is not allocated to the spare inthe deactivated state, the ICT 600 may transmit a signal forreallocating an address of the defective functional component to thespare to the system bus 500.

After monitoring whether the spare is in an idle state, the ICT 600 mayperform the test.

When there is no spare for the deactivated functional component, the ICT600 may allow the FPGA 900 to be programmed to imitate the same (orsimilar) operation as the deactivated functional component. Informationfor programming the FPGA 900 may be stored in the internal memory 400.Alternatively, the information for programming the FPGA 900 may bestored in a cache memory of the FPGA 900. Here, it can be understoodthat other ways of allowing the FPGA 900 to take over (or emulate) someor all of the operations of a deactivated component can be implemented.

As described above, when the FPGA 900 is programmed to imitate the sameoperation as the deactivated functional component, the ICT 600 maytransmit a signal including a request for updating an address table usedin the system bus 500. As an alternative, a signal including a requestfor reallocating the address of the defective functional component tothe FPGA 900 may be transmitted to the system bus 500. In other words,the existing address of the FPGA 900 may be revoked and replaced by anaddress of the defective functional component.

On the other hand, unlike in FIG. 6A, the FPGA 900 may lack a directconnection to the system bus 500, and may be connected to the system bus500 through a separate wrapper. That is, a separate wrapper may be addedbetween the system bus 500 and the FPGA 900.

When at least one functional component is determined as being defective,the SoC may be configured to display a warning message on a displaydevice which is communicable with the SoC.

When at least one functional component is determined as being defective,the SoC may be configured to transmit a warning message to a serverwhich is communicable with the SoC. Here, the server may be amanufacturer's server or a server in a service center. As describedabove, according to the present disclosure, the ICT and the wrapper arecombined in the SoC, and testing for defects may be performed during theruntime (or operation) of the SoC.

FIG. 6B is a view illustrating a first modified example of FIG. 6A.

Hereinafter, only the contents different from the description of FIG. 6Awill be described, and substantially the same contents will be referredto above-described content.

First, unlike in FIG. 6A, in FIG. 6B, the wrappers 700, that is, 700 a,700 b, 700 c, 700 d, 700 e, 700 f, and 700 g may be arranged orpositioned to be immediately adjacent to the system bus 500. In thiscase, the advantage is that SoC design and manufacturing can be muchsimpler. The implementation of this arrangement is not merely a simplematter of design choice, but numerous technical considerations need tobe taken into account. For example, arranging the wrappers next to thesystem bus has some technical hurdles. Also, there are manufacturingissues with respect to implementing such wrappers with the system bus.

As a specific example, as shown in FIG. 6B, the wrappers 700 may bearranged to partially or completely surround the system bus 500. Whenthe wrappers 700 are arranged to surround the system bus 500 in thisway, the system bus 500 can be safely isolated. Specifically, since thesystem bus 500 can be safely isolated by using the architecture orstructure as shown in FIG. 6B, it is possible to suppress or minimizeerroneous information/instruction from being transmitted to any one offunctional components through the system bus 500. In addition, since thesystem bus 500 can be safely isolated by using the architecture shown inFIG. 6B, even if any of functional components malfunctions, it ispossible to block the detrimental influence onto other functionalcomponents.

On the other hand, unlike FIG. 6A, the FPGA 900 may be not directlyconnected to the system bus 500, but is connected to the system bus 500through a wrapper 700 h. That is, a wrapper 700 h may be added betweenthe system bus 500 and the FPGA 900.

FIG. 6C is an exemplary view illustrating a second modified example ofFIG. 6A.

Hereinafter, only the contents different from the description of FIG. 6Aor FIG. 6B will be described, and substantially the same contents willbe referred to the above-described content.

First, unlike those shown in FIG. 6A or 6B, in FIG. 6C, each wrapper maybe arranged to partially or completely surround each functionalcomponent. In other words, each functional component may be implementedin the form of an independent island or dedicated section within thewrapper. In addition, a channel through which each functional componentis connected to the system bus is implemented in the form of a gatebridge, and the channel in the form of the gate bridge may be (partiallyor completely) opened and closed by each wrapper under the control ofthe ICT 600. Specifically, in the first wrapper 700 a, the NPU 100-1 maybe implemented in the form of an island. Similarly, the CPU 200 may beimplemented in the form of an island in the second wrapper 700 b. TheGPU 300 may be implemented in the form of an island in the third wrapper700 c. The internal memory 400 may be implemented in the form of anisland in the fourth wrapper 700 d. The memory controller 450 may beimplemented in the form of an island in the fifth wrapper 700 e. In thesixth wrapper 700 f, the I/O interface 800 may be implemented in theform of an island. The FPGA 900 may be implemented in the form of anisland in the seventh wrapper 700 h.

FIG. 6D is an exemplary diagram illustrating an exemplary architectureof a computing system including the NPU shown in FIG. 1 or FIG. 3.

Hereinafter, only the contents different from the description of FIG. 6Awill be described, and substantially the same contents will be referredto above-described content.

First, unlike that shown in FIG. 6A, in FIG. 6D, a plurality offunctional components, a system bus 500, ICT 600, and wrappers 700 maybe mounted on a board substrate 2000 rather than an SoC. The boardsubstrate may be a printed circuit board (PCB) or some other type ofsubstrate. Alternatively, the implementation shown in FIG. 6D may be asystem in package (SiP).

The plurality of functional components may include an NPU 100-1, acentral processing unit (CPU) 200, a graphics processing unit (GPU) 300,an internal (first) memory 400, a memory controller 450, a second memory470, an input output (I/O) interface 800, and a field programmable gatearray (FPGA) 900.

The first memory 400 can be implemented in various forms such as adynamic RAM (DRAM), a synchronous dynamic random access memory (SDRAM),a static RAM (SRAM), a magnetic random access memory (MRAM), a resistiverandom access memory (RRAM), and a high bandwidth memory (HBM). However,the present disclosure is not limited thereto.

The first memory 400 may include a memory controller. However, in FIG.6D, the memory controller 450 for the second memory 470 is shown toexist independently.

Similarly, the second memory 470 can be implemented in various formssuch as a dynamic RAM (DRAM), a synchronous dynamic random access memory(SDRAM), a static RAM (SRAM), a magnetic random access memory (MRAM), aresistive random access memory (RRAM), and a high bandwidth memory(HBM). However, the present disclosure is not limited thereto.

The first memory 400 or the second memory 470 may be configured to storeat least one of inference data, weight data, and feature map data of anartificial neural network model being processed by the NPU 100-1. Theinference data may be an input signal of an artificial neural networkmodel.

FIG. 6E is an exemplary view illustrating a first modified example ofFIG. 6D.

Hereinafter, only the contents different from the content describedabove will be described, and substantially the same contents will bereferred to above-described content.

First, unlike shown in FIG. 6D, in FIG. 6E, the wrappers 700, that is,700 a, 700 b, 700 c, 700 d, 700 e, 700 f, and 700 g may be arranged tobe positioned immediately adjacent to the system bus 500. In this case,it is advantageous that designing and manufacturing can be simplerand/or more cost effective.

As a specific example, as shown in FIG. 6E, the wrappers 700 may bearranged to at least partially surround the system bus 500. When thewrappers 700 are arranged to surround the system bus 500 in this way,the system bus 500 can be safely isolated. Specifically, since thesystem bus 500 can be safely isolated by using the architecture as shownin FIG. 6E, it is possible to suppress incorrectinformation/instructions from being transmitted to any one of functionalcomponent through the system bus 500. In addition, since the system bus500 can be safely isolated by using the architecture shown in FIG. 6E,even if any one of functional component malfunctions, it is possible toblock the influence on other functional components.

On the other hand, unlike FIG. 6D, the FPGA 900 is not directlyconnected to the system bus 500, but is connected to the system bus 500through a wrapper 700 h. That is, a wrapper 700 h may be added betweenthe system bus 500 and the FPGA 900.

FIG. 6F is an exemplary view illustrating a second modified example ofFIG. 6D.

Hereinafter, only the contents different from the description of FIG. 6Dor 6E will be described, and substantially the same contents will bereferred to above-described content.

First, unlike those shown in FIG. 6D or 6E, in FIG. 6F, each wrapper maybe disposed to surround each functional component. In other words, eachfunctional component may be implemented in the form of an independentisland within the wrapper. In addition, a channel through which eachfunctional component is connected to the system bus is implemented inthe form of a gate bridge, and the channel in the form of the gatebridge may be opened and closed by each wrapper under the control of theICT 600.

Hereinafter, for deeper understanding of the above-mentioned content, itwill be described in more detail with a table of contents.

I. Why Test During the Runtime is Important

In order to prevent or minimize potential accidents which may be causedby hardware defects in the autonomous computing system, various studieshave been conducted.

Among various tests, a pre-deployment test is included. According tothis test technique, all hardware designs are checked before selling theproduct to the clients. After the manufacturing, the design is testedfrom various viewpoints to detect and correct various problems which maybe found during the actual operation. For example, in order to test achip design, a test pattern is provided to perform the scanning of aninput and inspection for an output result. Even though this techniquemay minimize a potential problem for the hardware design before theshipment of the products, the problems of the defect during the runtimewhich may be caused due to the aging of the integrated circuits (ICs),external environments, and vulnerabilities of the complex designs cannotbe solved.

As described above, the above-described pre-deployment test cannoteffectively solve the hardware defects so that the inventor began to beinterested in test methods during the runtime.

From the viewpoint of a test mechanism, the pre-deployment test and thepost-deployment test seem to be similar, but there is an obviousdifference in when the test can be performed. Specifically, thepre-deployment test may be performed only at a specific time andgenerally may be allowed only shortly after the manufacturing. Incontrast, the test during the runtime may be performed at any time in anormal operation situation.

There may be two test techniques for the test during the runtimeincluding a function test and a scan test.

According to the function test, a test input is generated and an outputresult obtained by inputting the generated test input to an originaldesign is compared with an intended pattern. Alternatively, based on anoriginal design, according to the function test, input and outputsignals are monitored to detect an abnormality.

According to the scan test, architectures for the scan test are insertedinto the original design and various test patterns need to be created asmany as possible. As described, after preparing the scan architecturesand the test patterns, the test during the runtime may be performed invarious ways.

In order to perform the scan test, the ICT may connect the plurality offlip-flops in each CUT, inject the test input to at least one flip-flop,and acquire a test result from an operation of a combinational logic ofthe flip-flop to analyze whether the CUT is defective or normal duringthe runtime.

FIG. 7 illustrates an example of scanning a flip-flop structure.

In order to more easily design the hardware and minimize potentialmanufacturing defects, it is very important to apply a proper design fortestability (DFT).

To this end, an architecture for the scan test is reflected to thedesign and a test range with a specific ratio for all detectable defectsis defined to perform the test.

When a D-type flip-flop is used, the architecture for the scan test maybe easily reflected to the design. During the test, all flip-flops inthe CUT may operate as scan flip-flops including the D-flip-flops andmultiplexers.

As compared with the normal D-type flip-flop, as illustrated in FIG. 7,the flip-flop may use two additional pins, that is, a scan enable (SE)pin and a scan in (SI) pin. The SI pin is for test input and the SE pinenables the switching between an input (D pin) for a normal operationand a test input (SI) for a test operation.

FIG. 8 illustrates an example in which an architecture for scan test isadded in a hardware design.

As illustrated in FIG. 8, all SE pins in the scan flip-flop areconnected to the scan_enable (SE) ports and an SI pin of each flip-flopis connected to a Q pin of a previous flip-flop or a scan input port,and a Q pin of each flip-flop is connected to an SI pin of a subsequentflip-flop.

These connections create multiple scan chains. That is, the flip-flopsare threaded to each other to create a scan chain.

When the SE (scan_enable) port is enabled, all scan flip-flops transmitdata from the SI pin to the Q pin via the flip-flop and thus the datamay be transmitted from a scan_in port to a corresponding scan_out port.All the flip-flops on each scan chain shift the test input from thescan_in port to the scan_out port.

The smaller the number of flip-flops on the scan chain, the faster thespeed of shifting the data. However, the number of flip-flops on eachscan chain and the number of scan chains are dependent on each other.The more the scan chains created, the smaller the flip-flops on eachscan chain.

II. Test via ICT

The above-described test is performed as a background task so that thetest may be performed without degrading a system performance. Based onthe monitoring of an operation of a component to be tested, the ICT maydetermine whether the component is in an idle state. When the componentis in an idle state, the test is performed so that the degradation ofthe system performance may not be caused. The ICT consistently monitorsthe operation state of the CUT on the system bus and the CUT may respondto an unexpected access. When there is access to the CUT, an operationof the CUT is switched from a test operation to a normal operation torecover the CUT and come back the CUT to the normal operation. A slighttime delay may occur for the switching. According to the presentdisclosure, the system bus may be efficiently used during the time delayto minimize the degradation of the system performance due to therecovery.

II-1. Increase of Complexity of SoC or SiP Architecture

The design of the integrated circuit (IC) is becoming more complex dayby day, and the degree of integration is also increasing significantly.The SoC or SiP is a semiconductor device having a very high degree ofintegration so that the defects of some functional components may causethe degradation of the entire system performance. Accordingly, it isbecoming increasingly important to perform the test to find out thedefect of the functional components in the SoC.

FIG. 9A illustrates the SoC of FIGS. 6A through 6C or the system ofFIGS. 6D through 6F from a viewpoint of an operation.

The functional component (or IP) may be divided into three types: 1) aninternal processor, 2) an interface or a communication controller, and3) a memory. FIG. 9 shows a functional component (or IP) 100/200/300, aninternal memory 400, a memory controller 450, a system bus 500, an ICT600, a plurality of wrappers 700 a, 700 b, 700 c, 700 d, 700 f, and 700g (collectively denoted by 700), and an I/O interface 800.

The functional component (or IP) 100/200/300 may perform functionsrelated to encoding, decoding, encrypting, decrypting, and computing.The functional component (or IP) 100/200/300 acquires raw data from theinternal memory 400 and processes the data with a specific algorithm.When the processing is completed, output data may be transmittedtogether with a signal notifying the completion.

The internal memory 400 may be a read only memory (ROM) or a randomaccess memory (RAM). The ROM corresponds to a non-volatile memory andthe RAM corresponds to a volatile memory.

The volatile memory is a memory in which data is stored only when apower is supplied and the stored data is lost when the power isinterrupted. The volatile memory may include a static random accessmemory (SRAM) and a dynamic random access memory (DRAM).

The internal memory 400 may include a solid state drive (SSD), a flashmemory, a magnetic random access memory (MRAM), a phase change RAM(PRAM), a ferroelectric RAM (FeRAM), a hard disk, or a flash memory. Theinternal memory 400 may also include a synchronous random access memory(SRAM) and a dynamic random access memory (DRAM).

The I/O interface may support various protocols and functions to allowthe SoC or SiP to communicate with various external hardware.

However, when the SoC or SiP is built in an autonomous system to beused, the SoC or SiP may be compromised due to the aging of anelectronic element (e.g., a transistor), a physical impact, or a usageenvironment. Specifically, when the functional component (or IP) in theSoC handles important data, the compromised functional component maygenerate wrong output data, which may significantly deteriorate theaccuracy of the autonomous system.

In order to prevent this problem, as illustrated in FIG. 9A, the ICT 600monitors the system bus 500 and monitors states of the functionalcomponent 100/200/300, the internal memory 400, the memory controller450, and the I/O interface 800 via the wrapper 700 or the system bus500. When the functional component in an idle state is found, the ICT600 selects the functional component as a CUT to perform the test.

In FIG. 9A, a connection with the system bus during a normal systemoperation is represented with dotted lines and signals of the ICT andthe wrapper are represented with solid lines.

FIG. 9B illustrates an exemplary configuration for testing the NPU.

Referring to FIG. 9B, the NPU 100 may further include another componentto test the NPU. Specifically, referring to FIG. 9B, at least one of arandom number generator, a predetermined test data storage, a tempregister may be selectively further included in the NPU 100 as well asthe components illustrated in FIG. 1 or 3. The MUX (multiplexer) may bedisposed between the NPU internal memory 120 and the processing elementarray 110 to perform the internal test of the NPU 100. The MUX may beconfigured to switch a component configured to test the processingelement array 110 and the NPU internal memory 120.

A method of testing the processing element array 110 using a randomnumber will be described. The random number generator in the NPU 100illustrated in FIG. 9B may generate a random number based on apredetermined seed. At least one of the processing element arrays 110 isselected by the MUX to test whether the NPU 100 is defective.

The ICT 600 monitors the state of the NPU 100 via the wrapper 700 a andwhen the NPU 100 is determined to be in an idle state, the ICT 600 maycommand the NPU 100 to start the test.

As a specific example, the ICT 600 selects at least one of the pluralityPEs included in the NPU 100 to command to start the test.

As a specific example, when the ICT 600 determines that a certainpercentage of PEs (for example, 20% of PEs among all PEs) among theplurality of PEs included in the NPU 100 is in an idle state, the ICT600 may command the NPU 100 to start the test. In other words, when aratio of idle PEs among all the PEs is greater than or equal to athreshold value, the ICT may command to start the test.

As a specific example, when the ICT 600 selects a certain percentage ofPEs (for example, 50% of PEs among all PEs) among the plurality of PEsincluded in the NPU 100 to command the NPU 100 to start the test.

When the test of the NPU 100 is performed, the inference speed of theNPU, that is, inference per second (IPS) may be degraded. Specifically,the inference per second may be lowered in accordance with the number ofPEs to be tested. As a specific example, when 50% of PEs among all thePEs is tested, the inference per second may be lowered by approximately50% and when the 30% of PEs among all the PEs is tested, the inferenceper second may be lowered by approximately 30%.

Accordingly, according to examples, the NPU 100 may further includeextra PEs to improve the lowering of the speed due to the test.

As another example, when the NPU 100 operates at a value lower than apredetermined (or particular) IPS value, the ICT 600 may instruct theNPU 100 to perform the test. Specifically, when it is assumed that theNPU 100 operates at a maximum of 100 IPSs and a threshold IPS value is30 IPS, if the NPU 100 operates at 30 IPS or higher, the ICT 600 mayinstruct the NPU 100 to perform the test during the remaining time. Forexample, when the NPU 100 operates at 40 IPS, the test may be performedusing the remaining 60 IPS. Accordingly, the substantial speed loweringof the NPU may not be caused.

As another example, when the data which is transmitted to the NPUinternal memory 120 in the memory 400 is delayed so that the NPU 100 isin an idle state or enters a data shortage period, the ICT 600 mayinstruct the NPU 100 to perform the test.

When the test is performed on the NPU 100, a register file RFcorresponding to each PE in the NPU 100 is initialized to predeterminedtest input data and a corresponding PE may perform inference inaccordance with test input data in the register file RF. Thepredetermined test input data may be a function test of the NPU or apartial function test.

When the NPU 100 is tested, as described above, the random numbergenerator in the NPU 100 generates a random number. By doing this, theregister file RF is initialized by the generated random number and thecorresponding PE performs the inference in accordance with the randomnumber in the register file RF.

As an alternative, the ICT 600 commands the CPU 200 via the wrapper 700b to inject test input data into the register file RF in the NPU 100.

When the NPU 100 is tested, a plurality of register files RFs in the NPU100 is initialized to single test input data and a corresponding PE mayperform inference in accordance with test input data in the registerfile RF. Specifically, the plurality of PEs in the NPU 100 may be testedbased on the same single test input data and output the inferenceresult.

As an alternative, when the NPU 100 is tested, some register files RFsin the NPU 100 are initialized based on specific test input data and acorresponding PE may perform inference in accordance with test inputdata in the register file RF.

The register file RF may reset the flip-flops in each PE and transmitthe test input data to the PEs as described above.

For example, a size of each RF may be 1 Kb.

II-2. Necessity of Wrapper(s)

FIG. 10 illustrates an exemplary operation of a wrapper.

As described above, the ICT may test many functional components (thatis, IP, I/O interfaces, memories, etc.) in the SoC or SiP during theruntime of the SoC or SiP. To this end, during the test of thefunctional component selected as a CUT, a collision problem due to theaccess to the functional component from the system bus needs to besolved.

In order to solve the collision problem, after monitoring whether thefunctional component is in an idle state, when the functional componentis monitored to be in an idle state, the functional component isswitched from a normal operation mode to a test operation mode and thenthe test needs to be performed. When the collision is detected duringthe test, the functional component needs to be switched to the normaloperation mode. After switching the operation to the normal operationmode, the functional component needs to correctly process the inputdata.

To this end, the illustrated wrapper 700 needs to be disposed betweenthe functional components and the system bus 500. The wrapper 700 mayinclude multiplexer gates which selectively control the input and theoutput for each operation mode.

As illustrated in the drawing, when a TEST_ENABLE port is on, a testvector may be input to the CUT and a TEST_OUTPUT port may transmit theoutput. General data output from the wrapper 700 may be transmitted toother functional components via the system bus. In contrast, the testresult may be directly transmitted to the ICT 600. The ICT 600 mayreceive a test vector for the test from an external memory or aninternal memory and store the test result in the internal memory or theexternal memory or transmit the test result to the outside.

In order to test the SoC or SiP in the runtime, the ICT 600 may performa plurality of processes. First, the ICT 600 may select a functionalcomponent to be tested as a CUT based on a predetermined rule. Since theSoC or SiP is in the runtime, the CUT needs to respond to the accessfrom the system bus. Accordingly, it is effective to select a functionalcomponent in an idle state as a CUT as much as possible. To this end,the ICT 600 may monitor whether the functional component enters the idlestate. When the functional component enters the idle state, the wrapper700 may turn on the TEST_ENABLE port. The ICT 600 may inject the testvector to the CUT via the TEST_ENABLE port.

The ICT 600 may collect and analyze the test result from the CUT via theTEST_OUTPUT port of the wrapper 700. When the test result indicates thata problem is detected, the ICT 600 may perform a post action. During thetest, when a general access to the CUT from the system bus 500 isdetected, the ICT 600 may temporally delay the access from the systembus 500 and then may immediately stop (interrupt) the test operation.Thereafter, the ICT 600 may recover previous values for register settingof the CUT and turn off the TEST_ENABLE port of the wrapper 700. When anormal operation of the CUT is ready, the ICT 600 may control thewrapper 700 to return back the connection for input and output with theCUT to the system bus 500.

FIG. 11 illustrates an exemplary internal configuration of an ICT.

Referring to FIG. 11, the ICT 600 may include a configuration data(CONF_DATA) restorer 610, a state detector 620, a scheduler 630, atester 640, a test vector generator 650, a host interface 660, and apost action (POST_ACT) unit 670.

The state detector 620 may detect whether the functional components inthe SoC chip or SiP are in an idle state or a busy state (or aprocessing state). When an arbitrary functional component enters an idlestate, the state detector 620 transmits an ID (C_ID) of the functionalcomponent to the scheduler 630 to perform the test.

The scheduler 630 may manage an overall operation of the ICT 600. Thescheduler 630 may receive a state of the functional component from thestate detector 620 and trigger the test. The scheduler 630 may transmitthe ID of the component to the tester.

The tester 640 controls the wrapper 700, transmits a test vector,acquires a test result, and then compares whether the test resultmatches an intended test result. Thereafter, the tester 640 may transmitthe test result to the post-action unit 670. The tester 640 may restorethe register setting for the functional component selected as the CUT toits original value.

The test vector generator 650 may generate a test vector (or apredefined test input data) and a corresponding intended test result.The test vector generator 650 may include a buffer, a memory interface,a memory which stores the test vector and the intended test result, anda random number generator. When the test starts, a test pattern forgenerating the test vector may be loaded in the buffer. The randomnumber generator may be used to generate the test vector. The randomnumber generator may allow the memory not to store all the test vectors,but generate various test vectors.

When the ID (for example, C_ID) of the functional component from which aproblem is found is received from the tester 640, the post action unit670 may perform the post action. The post action may isolate thedefective functional component or notify a defect to the user or aremote host device.

The host interface 660 may report the functional component from whichthe problem is found during the test process to the user or the remotehost device. If there is a change related to the test operation, thehost interface 660 may notify the remote host device.

When the test is completed or the access to the functional componentselected as CUT from the system bus 500 is detected during the testprocess, the configuration data restorer 610 may restore the registersetting of the CUT to allow the tester 640 to switch the CUT to thenormal operation mode. Most of the functional components may have aspecific register setting value for a normal operation. Accordingly, theconfiguration data restorer 610 may store the register setting value ofthe functional component before performing the test and restore theregister setting value to the functional component when the CUT needs tobe switched to the normal operation mode.

II-3. To Detect Idle State of Functional Component

FIG. 12 illustrates an operation of monitoring whether a functionalcomponent is in an idle state by an ICT.

In order to detect whether the functional component is in an idle stateduring the normal operation mode, the ICT 600 may use one or both of twotechniques. First, the ICT 600 may monitor whether the component is inan idle state or is in use, based on some hardware signals whichdirectly or indirectly indicate whether to operate. For example, the ICT600 may monitor a power gating control signal to disconnect theconnection of the functional component to reduce the power consumptionof the functional component. Further, the ICT 600 may determine whetherthe functional component is in an idle state, based on an output signalwhich directly or indirectly indicates whether the component operates ora value of a register which stores information related to the operationin the functional component. Second, the ICT 600 monitors a signal froma system bus via the wrapper 700 or monitors an input/output port of thefunctional component during a specific time period to determine whetherthe functional component is in an idle state.

II-4. Processing of Access Collision(s)

FIG. 13 illustrates an operation between a master, a slave, and anarbiter which operate on a system bus.

The master on the system bus may be an entity which uses a slave, theslave may be an entity used by the master, and the arbiter may be anentity which performs arbitration and determination between the masterand the slave.

The slave illustrated in FIG. 13 may be a functional component selectedas a CUT and the arbiter may be an ICT.

When an access for a normal operation is detected from the system bus500 while the functional component selected as a CUT is being tested,the ICT 600 may require a predetermined amount of time or more torecover the CUT to its previous state. The ICT 600 may temporallydeactivate (or de-assert) an HREADY signal to temporally stop the systemaccess from the master, stop (interrupt) the test activity, recover theregister setting of the CUT, and change a direction of data which isinput to or output from the wrapper. When the CUT which is the slave isready to perform the task with the master, the HREADY signal may beturned on. However, according to the present disclosure, the ICT mayinduce some time delay for a bus separation operation. A specificprocess will be described below.

First, the master activates (or asserts) an HBUSREQ signal for a busaccess. Second, during the arbitration or determination process, thearbiter activates (or asserts) an HGRANT signal to allow the bus access.By doing this, the master may transmit the data to the CUT which is aslave via the system bus. If the ICT is performing a processingoperation for a test, the ICT transmits an HSPLIT signal to the arbitertogether with a bit indicating a current master and activates (orasserts) an SPLIT signal in the HRESP signal, simultaneously. After theactivation (assertion), the master nullifies the access to the CUT andthe arbiter performs the arbitration or determination process withouthaving intervention of the master. When the CUT is ready to respond tothe access from the master, the ICT deactivates the HSPLIT signal andthe master waits for a grant from the arbiter to resume the task toaccess the CUT.

FIG. 14 illustrates an example in which a shift register is added in anSoC chip or SiP.

The inventor of the present disclosure has recognized that the access tothe I/O interface may not cause the collision on the system bus. Forexample, when the target CUT is a master, an external device connectedthrough the I/O interface does not request the access for itself so thatthe collision may not occur. Accordingly, it may be effective to focusonly on solving the collision problem generated when the CUT is a slave.

Instead, in order to delay data which is transmitted from the externaldevice to the CUT during the restoring time, a shift register may beadded between a port of the SoC and the external interface port of theCUT.

The shift register may be added to store the access signal input fromthe outside of the SoC while the CUT is restored. When the CUT is ready,the access signals are regenerated by the shift register to be output.

A depth of the shift register may be determined by the number of clockcycles required to restore the CUT to a normal operation. Specifically,when one or more functional components need to receive a signal from theoutside of the SoC, the depth of the shift register may be variable. Inthis case, the depth of the shift register may be determined by the ICT.

II-5. Operation Order (or Sequence) of ICT

FIG. 15 illustrates an operation order of an ICT.

Referring to FIG. 15, when a timer related to the test start of the ICTin the runtime expires (S601), the ICT monitors whether an arbitraryfunctional component is in an idle state and detects a functionalcomponent in an idle state (S603).

By doing this, the ICT performs a test preparation process (S605). Thetest preparation process may include selecting the functional componentas a CUT, isolating the functional component selected as a CUT from thesystem bus, and generating a test vector as test input data. Theisolation from the system bus may mean that the ICT changes thedirection of the input and the output on the wrapper which communicateswith the functional component selected as the CUT.

The ICT injects the test vector which is the test input data into theCUT (S607).

When the test is normally completed, the ICT checks the test result(S609). For the checking, the ICT may compare whether the test resultmatches the intended test result.

When the test result indicates that there is no problem in thefunctional component selected as the CUT (that is, no defect or damage),the ICT may recover the functional component to a normal operation state(S611).

In the meantime, when an access to the functional component selected asthe CUT is detected from the system bus during the test preparation orthe test, the ICT may recover the functional component selected as theCUT to a normal operation state (S613). The recovery may mean that aregister setting value of the functional component selected as the CUTis recovered and the direction of the input and the output returns to anoriginal state on the wrapper which communicates with the functionalcomponent selected as the CUT.

In this case, the ICT drives a back-off timer (S615) and when theback-off timer is expired, may return to the step S603.

In the meantime, when the test result indicates that there is a problemin the functional component selected as the CUT (that is, the defect ordamage), the ICT may perform the post-operation (S617).

II-6. Test for Internal Memory

FIG. 16 illustrates a test process of an internal memory.

The test for the internal memory may be different from the test for thefunctional component. Hereinafter, two test techniques for the internalmemory will be proposed.

A first technique is a technique of detecting an error using an errordetection code during a process of reading data from the internalmemory. If an error detection code acquired during the reading processis different from a predetermined error detection code, the ICT maydetermine the code as an error.

A second technique is a technique of performing a read-write test in ahard way during a normal operation.

FIG. 16 illustrates the second technique. A test logic which enclosesthe internal memory may perform the read-write test during the runtimeof the system and bypass the access from the system bus. In order tocompletely process the test, the tester in the ICT may be responsiblefor the address management. The illustrated temporally register file maytemporally store original data which is prone to be deleted due to thetest. When the test is completed, the original data in the temporaryregister file may be recorded in the internal memory again.

If an unpredictable access occurs during the test, data on the systembus may be recorded in the temporary register file, and in contrast, thedata in the temporary register file may move to the system bus.

The test technique as described above may be applied not only to theinternal memory, but also to the external memory in the same way.

II-7. Operation after Testing

When there is a hardware defect in the SoC or SiP, the operation afterthe test may be very important. For example, a user is notified of thedefect in order to recommend stopping usage. To this end, the postaction unit 670 of FIG. 11 may provide information about the functionalcomponent from which the defect is detected and information about testinput data (that is, a test vector) which causes the defect. Theabove-described information may allow the user to know the position ofthe defective functional component. The usage of the functionalcomponent from which the defect is detected needs to be stopped andisolated. In order to prevent the defective functional component fromdegrading the performance of the entire system, the output signal of thefunctional component may be replaced by a predetermined signal.Alternatively, the functional component may be reset or gated.Alternatively, the power gating may be performed on the functionalcomponent.

In the meantime, when the functional component is isolated, the SoC orSiP may face another problem. Therefore, even though some functionalcomponents have defects, a method for allowing the SoC or SiP to stilloperate needs to be proposed. For example, when the SoC or SiP ismounted in a product which requires a high reliability, the SoC or SiPneeds to further include a spare for some functional components. If somefunctional components have defects, the spare may operate instead of thefunctional component. However, when some functional components areduplicated, it may increase an area of the semiconductor device. Inorder to solve this problem, it may be effective to add a programmablelogic in the SoC or SiP.

III. Function Test During SoC or SiP Runtime or Test for Combination ofFunctions

FIG. 17 illustrates a process of testing a function using a randomnumber generator.

The function test is a test of injecting test input data (for example, atest vector) into a CUT and comparing whether an output from the CUTmatches an intended output. In order to correctly evaluate based on thecomparison, each input data needs to accurately induce an intendedoutput. A test range of the test input data needs to be high to detectall defects.

In a specific design, there may be two test input data for the functiontest. First, a random number generator which is connected to an XORoperation may be used for the test operation illustrated in FIG. 17.Generally, the random number generator may generate a pseudo randomnumber stream based on an input seed. The random number stream isinjected into the CUT via the wrapper and the output is accumulated andstored in the test result register by means of the XOR operation. Whenthe test is completed, the values stored in the test result register maybe compared with the intended result corresponding to the test inputdata. If there is a difference in the comparison result, an errornotification may be issued.

Second, all test patterns for test input data and correspondingprediction results may be fixed, respectively and stored in the internalmemory in the SoC or SiP or an external memory. When the test input data(that is, a test vector) from the memory is input to the CUT, the outputfrom the CUT and the intended result corresponding to the test inputdata may be compared.

In order to perform the function test during the runtime of the SoC orSiP, the ICT plays an important role to transmit data and communicatewith the system bus, and monitor the state of the CUT. Specifically,when the CUT is in an idle state, the ICT needs to determine when thetest is performed. During the test, the random number generatorgenerates a random number stream as test input data and transmits thetest input data to the CUT. If there is a difference between the testresult and the intended test result, the ICT transmits the informationto the post action unit.

During the function test, the functional components in the SoC may beused so that generally, a frequency for the test operation needs to belower than or equal to a frequency for a normal operation to avoid thedifference of the timing (that is, timing violation). In order toperform the test in real time during the normal operation, it iseffective to perform the test when the functional component is in anidle state. Therefore, there is no choice but to perform the test at ahigh frequency.

IV. Test in Runtime of SoC or SiP Using Combination of DFT (DiscreteFourier Transform) and ICT IV-1. Multiple Clocks

FIG. 18A illustrates an example of multiple clocks, FIG. 18B is anexemplary view illustrating an operation of a tester under the multipleclocks, and FIG. 18C illustrates a path of test input data.

During the test, with regard to the injection of one test input data(that is, a test vector), there may be two techniques.

A first technique is to use a time period to “shift data” as illustratedin FIG. 18A. An SE (scan enable) port is enabled, and the Q output of aflip-flop is connected to the D input of another flip-flop. Thisconnection may make a scan chain which connects the scan input to thescan output through a chain of the flip-flops.

Therefore, all the designed combinational logics may be disabled andthere may be no reference logic cell for a data path (that is, a pathfrom one flip-flop to another flip-flop).

When T_(cycle) is defined as a clock cycle, T_(launch) is defined as atime delay from a clock source of a first flip-flop to a CK pin,T_(capture) is defined as a time delay from the clock source to a CP pinof a second flip-flop, T_(clk2q) is defined as a time delay from a CK ofthe first flip-flop to a Q pin, and T_(dpmax) is defined as a time delayfrom Q of the first flip-flop to D of the second flip-flop,T_(cylce)>T_(launch)+T_(clk2q)+T_(dpmax)+T_(setup)+T_(margin)−T_(capture).When the scan test is enabled, T_(dpmax), may be reduced to zero from aviewpoint of the scan test. Ideally, T_(dpmax) may be zero. However, inorder to solve the timing violation, when a plurality of inverters orbuffers is added, the time delay may be larger than zero.

As an alternative,T_(dpmax)>>T_(clk2q)+T_(setup)+T_(launch)−T_(capture). During a timeperiod to “shift data,” it may be processed at a higher frequency.

During a time period to “capture data” as illustrated in FIG. 18A, ascan enable pin is deactivated and thus the functional component isre-activated and a combinational logic may be activated on the datapath. In order to solve the violation of the timing while data iscaptured, a time delay may be added between a clock located at one endin a time period to “shift data” and a clock located at one end in atime period to “capture data.”

The delay between clock cycles may be greater than or equal to a clockcycle for a normal operation. In order to detect when the time period to“shift data” is completed based on a maximum number of flip-flops on thescan chain corresponding to a shifted value, a counter is added and inorder to manage the time delay in a time period to “capture data,”another counter may be added.

In FIG. 18B, a test block receives two input clocks. One is f_clk usedfor a normal operation and the other one is sclk to “shift data.” A“clock configuration” is inserted into the tester block, so that s_clksignal may be set to be used in both the period to “shift data” and theperiod to “capture data.”

In order to control the switching between f_clk for a normal operationand s_clk for test operation, a TE signal corresponding to the CUT maybe used. When an ID (that is, C-ID) of the component is received fromthe scheduler, the test block in the ICT is ready to test. TEs of theCUTs which are available through the decoder may enable the testprocess.

FIG. 19A illustrates an example of a functional component, and FIG. 19Billustrates an example that test input data (for example, a test vector)is injected into a tester in an ICT.

In order to apply a discrete Fourier transform (DFT) in the test duringthe runtime of the SoC or SiP, a scan chain is added in the CUT and allflip-flops may be enclosed by the scan flip-flop. A scan input, a scanoutput, and TEST_ENABLE, and SCAN_ENABLE signaling are connected to thetester in the ICT and an original input and an original output of theCUT may communicate with the system bus via the tester and the wrapper.

As illustrated in FIG. 19B, from a viewpoint of the memory which storesthe test pattern, the block may be divided into four parts. A first partis a part which stores an input shift vector, a second part is a partwhich stores an output shift vector, a third part is a part which storesan input capture vector, and a fourth part is a part which stores anoutput capture vector. In order to start the test, the input shift datais loaded from the memory to input to the CUT through the tester.

In each scan chain, after all the flip-flops are filled with the shiftvector, when the first input capture vector including a value for a scaninput and an initial input is loaded, a first output capture vectorincluding values for all scan outputs and initial outputs is loaded, andthen compared with the actual output capture data. Each loaded shiftvector is accompanied by output shift data and the actual output dataand the output shift vector or an output capture vector may be compared.

FIG. 20 illustrates a test process using a DFT, and FIG. 21 illustratesan example of shift data and capture data during a test process.

During a step of shifting data, when a scan_enable port is enabled, aSCAN_IN port may be connected to the SCAN_OUT port through theflip-flops without the combinational logic. An input shift vector may beloaded in all scan chains until all flop-flops have values shifted fromthe input shift vector. One shift value may pass through one flip-flopat each clock cycle. That is, a D pin of a previous flip-flop may beconnected to a D pin of a subsequent flip-flop.

When during a capturing step, a scan_enable port is disabled, D pins ofall flip-flops are not connected to the Q pins of the previousflip-flops, but may be directly connected to the combinational logic.

The capture vector output may be loaded in the Q output of all theflip-flops through the combinational logic at the positive (+) edge of aclock cycle. In a first data capturing step, a data transmitting processis prepared to compare the output data with intended output data andthen the comparison is performed at every positive clock edge. All testvector inputs are loaded and the process returns to the first data shiftstep and each process starts over.

FIG. 21 illustrates shifting and capturing processes. A rectangular boxin FIG. 21 indicates a flip-flop in each scan chain, and all flip-flopsare filled at the end of the data shifting step.

FIG. 22 illustrates an example of switching a test mode to a normaloperation mode.

As known with reference to FIG. 22, a data shifting process and acapturing step may be repeated during the output test mode. If there isan access to the CUT, the CUT is recovered to a normal operation modeand the test may be backed off. Thereafter, the skip mode is performedduring a predetermined time period and then the output test mode may beperformed again.

FIG. 23 illustrates an example that flip-flops operate on a scan chain,and FIG. 24 illustrates a part of a CUT which operates in a normaloperation mode.

When an unexpected access to the CUT from the system bus is generated,TEST_ENABLE is disabled and the data shifting or capturing may bequickly stopped. The CUT is recovered to a normal operation mode and thetest may be backed off.

When the CUT enters an idle state again, a previous data shift step maystart over for the test. However, in a first shifting step aftershifting from the normal operation mode to the test operation mode, thecomparison of the output result is deactivated and the comparison of theoutput results may be performed from the subsequent capturing step.

That is, as illustrated in FIGS. 23 and 24, the shifted input values arenot loaded in all the flip-flops at the scan chain, and the comparisonmay not be performed.

V. Simulations

In order to verify the above-described contents, simulations wereperformed from simple sub-cases to complex sub-cases using an electronicdesign automation tool of Synopsys, Inc. When the verification issuccessfully performed, a design implemented by software codes may beconverted into a logic gate. As a next step, all D-flip-flops (DFFs) maybe replaced by the scan flip-flop and scan chains may be generated. Inorder to increase a range covered by the test, Netlist may be repeatedlymodified and tested. The ATPG tool may be used to generate the testvector and the intended result. When a scan and a test pattern intowhich the Netlist is inserted are prepared, the ICT may be applied tothe design. Details of each test case will be described with referenceto the drawings.

FIG. 25 illustrates a process for simulation.

The illustrated simulation process uses a design compiler tool ofSynopsys, Inc. A design compiler (DC) too may be used to convert asoftware code into a logic gate level based on a timing restriction suchas a period, conversion, capacitance, timing information included in alibrary package. When all restrictions are satisfied, the valueoptimization may be repeatedly performed. When the design does notsatisfy the requirement, the restriction may be adjusted.

The output of the DC tool including the above-described timingrestriction may be used as an input of a design for test (DFT).

In a scan test injecting step, the number of scan ports and a scan chainmay be set. Generally, in order to minimize an additional port duringthe design, original input and output ports are used to generate a scanport. Further, the number of input scan ports is equal to the number ofscan chains. The larger the number of scan chains, the smaller thenumber of shift clock cycles to shift the data. Accordingly, themaximizing of the number of scan chains is the best option for the test.When the scan setting is completed, the DFT compiler may replace all theflip-flops with the flip-flops for scanning and connects the scan input(scan_in) pin and a scan out (scan_out) pin to each other to generate ascan chain. Additional connection and the scan flip-flops make thedesign more complex and cause the time delay in most of the data path sothat DFT compiler may continuously optimize the power and the timingafter connecting the scan chains. After completing the scan testinjection, the DFT DRC which follows the DFT rule checks whether alltest connections are connected. The test input data (that is, a testvector) is ready.

In order to check the test range and generate the test pattern, theoutput of the DFT compiler is input to Tetramax of Synopsys, Inc. Whenthe test range does not satisfy the intended requirement, the scan testinjecting step may be performed again to modify the design. This taskmay be repeated until a desired test range is acquired.

V-1. Experiment for Design

As an experiment for design, a JPEG image encoder may be used. This testmay use approximately 265,118 combinational cells, 72,345 sequentialcells, and 31,439 inverter/buffer cells. As a result of performing basedon cell library information and many threshold values, it was confirmedthat a frequency which satisfies the timing restriction was 100 MHz anda frequency for shifting the test pattern was 1 GHz. Approximately 512scan chains were used and the maximum number of flip-flops used at eachscan chain was 75. Accordingly, it was confirmed that a time period usedto shift the test pattern was approximately 75 cycles which correspondto 75 ns. One cycle is consumed to capture the data, which correspondsto approximately 10 ns. Approximately 256 test patterns are input andeach test pattern includes approximately 75 test vectors for shiftingand approximately one test vector may be used for capturing. In order tocomplete one test period, 13,260 (156×75+156×10) ns was taken.

FIG. 26 illustrates a test architecture for a JPEG image encoder.

As described above, in order to check whether a ratio of controllableand observable nodes is sufficient for the test, Tetramax is used toconfirm the test range. Generally, 99.97% of test range and 1,483.5 Kbytes were used for the test.

From the viewpoint of power consumption, each test was measured bydifferent types of inputs. In the following Table 1, an internal power,a switching power, and a leaked power for each input are represented.Specifically, inputs which cannot be controlled for the test mode andthe normal mode were implemented first. This is called static powermeasurement. After inputting to the power compiler tool, in order toestimate the power consumption in the test mode, a TE (test_enable) portwas turned on and in order to estimate the power consumption in thenormal operation mode, the TE (test_enable) port was turned off. Second,in order to estimate the power consumption, the inputs were controlledby a specific time interval. This is called dynamic power estimation.The controlling of inputs may be classified into three modes. First, thetest mode is turned on and the test pattern is provided. Second, thetest mode is turned off, and an input for the normal operation isprovided. Third, after completing the test for each test pattern, themode is switched to the normal operation mode. In order to obtain apower consumption value therefor, the test mode and the normal operationmode are switched.

The following table is a power consumption value of a JPEG imageencoder.

TABLE 1 Uncontrollable input Control input (114,360 ns) Type of powerNormal Normal consumption Test mode operation mode Test mode operationmode Switch_F_S Switching 265.4368 3.4659 945.5015 76.6824 622.9294power (mw) Internal 487.6708 2.9630 1.5978e+03 109.4660 1.0651e+03 power(mw) Leakage 4.05 4.0645  4.0479 4.058  4.0502 power (mw) Total 757.158810.4934 2.5474e+03 190.2058 1.6921e+03 power (mw)

Similar to the JPEG image encoder, a functional component for anadvanced encryption standard (AES) was tested. Further, the functionalcomponent for image classification in an autonomous vehicle was alsotested. The results are represented below.

The following Table 2 represents an AES design.

TABLE 2 Number of combinational cell 160,261 Number of sequential cells11,701 Number of buffers/inverters 22,377 Entire area 400,464.797234Frequency of normal operation mode 100 Mhz Frequency of test mode 1 GhzTshift 46 ns Tcapture 10 ns Number of test patterns 315 Range of test100% Memory size 948.5 KBytes

The following Table 3 represents a test for a functional component forAES.

TABLE 3 No control Control input input (402 ns) Normal Power NormalNormal operation estimate Scan operation Scan operation mode & (mW) modemode mode mode Test mode Switching 59.7507 23.0733 213.8727 1.7887135.5754 power Internal 106.2405 32.7560 307.6255 2.6223 192.9561 powerLeakage 1.3456 1.3455 1.3456 1.3516 1.3459 power Total power 167.337557.1769 522.8372 5.7625 329.8691

The following Table 4 represents details of CONVO2.

TABLE 4 Number of combinational cells 2,245,932 Number of sequentialcells 424,695 Number of buffers/inverters 154,510 Frequency of normaloperation mode 50 Mhz Frequency of test mode 1 Ghz Tshift 829 nsTcapture 20 ns Number of test patterns 183 Range of test 100% Memorysize 18.634 Mbytes

The following Table 5 represents power consumption for CONVO2.

TABLE 5 Power No control input estimate Normal (mW) Scan mode operationmode Switching power 3,721.1 390.9510 Internal power 5,759.5 633.5883Leakage power 0.427 0.426 Total power 9,540.8 1,070

The function test and the test through the scan injection haveadvantages and disadvantages. The test through scan injection has adisadvantage in that more memories are used as compared with thefunction test and the time is delayed and has an advantage in that therange of the test is broad.

Specifically, when the SoC or SiP is mounted in a product which requiresa high reliability like an autonomous vehicle, a drone, an urban airmobility (UAM) vehicle, an unmanned aerial vehicle (UAV), a scaninjection type test having a broad test range may be advantageous.Further, the scan injection type test may increase a frequency for atest operation and reduce the test time. When it takes a long time totest, the possibility of the car accident may be increased so that it isnot desirable. The scan injection type test may increase a frequency fora test operation so that more test patterns may be injected during theidle time and the hardware defect in the SoC may be more quicklydetected. The normal function test has an advantage in that the powerconsumption is small, but in the environment in which the highreliability is required, such as the autonomous vehicle, a drone, anurban air mobility (UAM) vehicle, or an unmanned aerial vehicle (UAV),the power consumption does not matter.

The SoC has been mainly explained so far, but the disclosure of thepresent specification is not limited to the SoC and the contents of thepresent disclosure are also applicable to a system in package (SIP) or aprinted circuit board (PCB)-based board level system. For example, eachfunctional component is implemented by an independent semiconductor chipand is connected by a system bus which is implemented by an electricallyconductive pattern formed on the PCB.

The examples of the present disclosure disclosed in the presentspecification and the drawings merely provide a specific example foreasy description and better understanding of the technical descriptionof the present disclosure, but are not intended to limit the scope ofthe present disclosure. It is obvious to those skilled in the art thatother modifications are possible in addition to the examples describedso far.

What is claimed is:
 1. A system capable of testing a component in thesystem during runtime, the system comprising: a substrate; a pluralityof functional components on the substrate and at least one of thefunctional components including circuitry; a system bus in a form ofelectrically conductive patterns on the substrate that allow theplurality of functional components to communicate with each other; oneor more test wrappers, each of the one or more test wrappers connectedto at least one of the plurality of functional components; and anin-system component tester (ICT) configured to: select, as a componentunder test (CUT), at least one functional component, in an idle state,of the plurality of the functional components; and test, via the one ormore test wrappers, the at least one functional component selected asthe CUT.
 2. The system of claim 1, wherein the ICT is further configuredto: monitor, via the one or more test wrappers, one or more states ofthe plurality of the functional components.
 3. The system of claim 1,wherein the ICT is further configured to: stop the test with respect tothe at least one functional component, based on a detection of acollision with an access from the system bus to the at least onefunctional component; and allow a connection of the at least onefunctional component to return back to the system bus, based on thestop.
 4. The system of claim 3, wherein the ICT is further configuredto: return to the select step, if the at least one functional componentis in the idle state again, after the connection of the at least onefunctional component is returned back to the system bus.
 5. The systemof claim 4, wherein the returning to the selection step occurs after aback-off timer with respect to the collision expires.
 6. The system ofclaim 1, wherein the plurality of functional components include: one ormore universal processing units (UPUs).
 7. The system of claim 6,wherein the UPU includes: one or more central processing units (CPUs);one or more graphic processing units (GPUs); and one or more neuralprocessing units (NPUs) configured to perform operations for anartificial neural network (ANN) model.
 8. The system of claim 6, whereinthe plurality of functional components further include one or more of:at least one memory; at least one memory controller; and at least oneinput and output (I/O) controller.
 9. The system of claim 1, wherein theICT is further configured to: instruct, for the testing step, the one ormore test wrappers to isolate from the system bus a connection of the atleast one functional component.
 10. The system of claim 1, wherein theICT includes at least one or more of: a detector configured to monitorone or more states of the plurality of the functional components; ascheduler configured to manage operations of the ICT; a generatorconfigured to generate test input data; and a tester configured toinject the test input data into the CUT and analyze a test resultacquired from the CUT processing the test input data.
 11. The system ofclaim 10, wherein the test input data is a predefined test data orrandom bit streams which are generated based on a seed.
 12. The systemof claim 1, wherein the ICT is further configured to: perform ananalysis of a test result acquired from the at least one functionalcomponent, after the testing step is completed; and allow a connectionof the at least one functional component to the system bus or to anothersystem connection, based on the at least one functional component foundto be normal after the analysis.
 13. The system of claim 1, wherein theICT is further configured to: deactivate the at least one functionalcomponent, if the at least one functional component is found to bedefective.
 14. The system of claim 1, further comprising: a fieldprogrammable gate array (FPGA) configured to imitate the at least onefunctional component that is found to be defective.
 15. The system ofclaim 14, wherein the FPGA's address is revoked and replaced by anaddress of the at least one functional component that is found to bedefective.
 16. The system of claim 13, wherein the deactivating stepincludes revoking an address of the at least one functional componentthat is found to be defective or includes powering off the at least onefunctional component that is found to be defective.
 17. The system ofclaim 13, wherein the deactivating step includes isolating the at leastone functional component being analyzed as defective from the system busby cutting off a system bus connection to the at least one functionalcomponent that is found to be defective.
 18. The system of claim 11,wherein the plurality of functional components includes a sparecomponent that can replace the at least one functional component that isfound to be defective; and wherein the ICT is further configured toactivate the spare component.
 19. The system of claim 1, wherein thetesting step includes a scan test, which is different from a functiontest, and wherein, for the scan test, the ICT is further configured to:connect a plurality of flip-flops in each CUT to each other, inject testinput into at least one flip-flop, and acquire a test result fromoperations of combinational logics of the flip-flops to analyze whetherthe CUT is defective or normal during runtime.
 20. The system of claim1, wherein the plurality of functional components include a neuralprocessing unit (NPU), the NPU including a plurality of processingelement arrays and being configured to select and test at least oneprocessing element of the plurality of processing element arrays. 21.The system of claim 1, wherein a plurality of the test wrappersstructurally isolate the system bus from signal interference.
 22. Anapparatus comprising: a plurality of functional components respectivelyconnected to a system bus via a respective one of a plurality of testwrappers; and an in-system component tester (ICT), operatively connectedto the system bus and to the test wrappers, configured to performoperational testing of at least one functional component to determinewhether the function component is normal or defective, wherein theoperational testing is performed during an operational run-time of theapparatus, wherein the system bus, the test wrappers and the ICT aremounted together on a same substrate or a same board.
 23. The apparatusof claim 22, wherein, if the at least one functional component isdetected to be in an idle state during the operational run-time of theapparatus, the functional component is isolated from the system bus, andthe ICT performs the operational testing on such isolated functionalcomponent.
 24. The apparatus of claim 23, wherein if the functionalcomponent is found to be not defective, the functional component isreturned to the system bus.
 25. The apparatus of claim 23, wherein ifthe functional component is found to be defective, such defectivefunctional component is kept isolated from the system bus.
 26. Theapparatus of claim 25, wherein the plurality of functional componentsinclude a spare component that is capable of being activated when thefunctional component is found to be defective as a result of theoperational testing.
 27. The apparatus of claim 25, further comprising:a field programmable gate array (FPGA), operatively connected to thesystem bus and to the ICT, capable of being adapted to perform at leastsome functionalities of the functional component that is found to bedefective as a result of the operational testing.
 28. The apparatus ofclaim 22, wherein the plurality of the test wrappers have an arrangementthat provides isolation for the system bus.